Nov 03 21:29:11.073809 ip-10-0-137-212 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 21:29:11.487643 ip-10-0-137-212 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 21:29:11.487643 ip-10-0-137-212 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 21:29:11.487643 ip-10-0-137-212 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 21:29:11.487643 ip-10-0-137-212 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 21:29:11.493584 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493477    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:11.493584 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493577    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:11.493584 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493581    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:11.493584 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493584    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:11.493584 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493587    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:11.493584 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493591    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493595    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493598    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493601    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493603    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493606    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493609    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493612    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493614    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493617    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493620    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493622    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493625    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493628    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493630    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493633    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493635    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493638    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493640    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493643    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:11.493805 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493645    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493650    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493656    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493660    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493663    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493666    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493668    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493671    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493674    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493676    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493679    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493681    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493684    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493687    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493691    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493693    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493696    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493699    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493701    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:11.494273 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493704    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493707    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493710    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493712    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493715    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493718    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493720    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493723    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493725    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493728    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493731    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493734    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493736    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493739    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493741    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493744    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493747    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493750    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493752    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:11.494734 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493756    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493761    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493765    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493769    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493771    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493774    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493778    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493781    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493785    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493787    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493790    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493793    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493795    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493798    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493801    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493803    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493806    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493809    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493811    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493815    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:11.495179 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493818    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493821    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493824    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493826    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493829    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493831    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493834    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493837    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493839    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493842    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493844    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493847    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493849    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493852    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493855    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493857    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493860    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493863    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493865    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493868    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:11.495696 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493873    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493876    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.493879    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494704    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494713    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494717    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494721    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494723    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494726    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494729    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494732    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494734    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494737    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494745    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494748    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494751    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494754    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494756    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494759    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:11.496236 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494761    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494764    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494766    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494769    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494772    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494774    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494777    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494780    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494782    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494785    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494787    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494789    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494792    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494796    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494798    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494801    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494804    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494807    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494810    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494812    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:11.496747 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494815    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494817    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494820    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494823    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494825    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494828    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494830    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494833    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494837    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494842    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494845    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494848    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494850    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494853    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494856    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494858    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494860    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494863    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494866    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:11.497214 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494869    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494871    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494875    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494877    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494879    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494883    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494885    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494888    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494891    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494894    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494896    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494899    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494901    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494904    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494906    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494909    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494911    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494914    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494917    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494919    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:11.497720 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494921    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494924    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494927    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494929    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494932    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494934    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494937    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494939    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494942    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494944    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494947    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494950    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494953    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494955    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494958    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494960    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494963    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494966    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494969    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494971    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:11.498203 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494974    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494976    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494979    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494981    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494985    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494987    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494991    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494993    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494996    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.494998    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495001    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495524    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495529    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495533    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495536    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495539    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495542    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495545    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495547    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:11.498691 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495550    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495553    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495555    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495558    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495562    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495564    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495567    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495570    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495573    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495575    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495578    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495581    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495583    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495586    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495588    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495591    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495594    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495596    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495599    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495601    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:11.499153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495604    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495606    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495609    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495611    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495614    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495617    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495619    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495622    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495625    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495628    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495635    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495637    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495640    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495643    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495646    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495648    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495651    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495653    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495656    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495658    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:11.499651 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495661    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495663    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495666    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495669    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495673    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495676    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495679    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495682    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495685    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495687    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495691    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495693    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495696    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495698    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495702    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495704    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495707    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495710    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495713    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:11.500113 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495716    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495719    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495722    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495725    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495728    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495730    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495733    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495736    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495738    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495741    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495743    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495746    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495749    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495753    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495757    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495760    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495763    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495766    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495768    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:11.500574 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495771    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495774    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495776    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495779    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495781    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495784    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495786    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495789    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495791    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495794    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495796    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495799    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495801    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495804    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495806    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495809    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495813    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495817    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495820    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.495822    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:11.501029 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.496973    2580 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.496983    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.496992    2580 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.496997    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497002    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497005    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497010    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497014    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497018    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497021    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497025    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497028    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497031    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497035    2580 flags.go:64] FLAG: --cgroup-root=""
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497038    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497041    2580 flags.go:64] FLAG: --client-ca-file=""
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497044    2580 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497047    2580 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497051    2580 flags.go:64] FLAG: --cluster-domain=""
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497053    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497057    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497060    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497063    2580 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497067    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 21:29:11.501486 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497071    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497074    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497077    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497080    2580 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497085    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497088    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497093    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497096    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497101    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497104    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497107    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497110    2580 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497113    2580 flags.go:64] FLAG: --enable-server="true"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497116    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497122    2580 flags.go:64] FLAG: --event-burst="100"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497125    2580 flags.go:64] FLAG: --event-qps="50"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497128    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497131    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497134    2580 flags.go:64] FLAG: --eviction-hard=""
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497138    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497141    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497144    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497147    2580 flags.go:64] FLAG: --eviction-soft=""
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497151    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497154    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 21:29:11.502063 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497157    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497160    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497163    2580 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497166    2580 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497169    2580 flags.go:64] FLAG: --feature-gates=""
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497173    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497176    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497180    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497183    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497186    2580 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497190    2580 flags.go:64] FLAG: --help="false"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497193    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-137-212.ec2.internal"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497197    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497202    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497205    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497209    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497212    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497215    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497218    2580 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497221    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497224    2580 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497228    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497231    2580 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497234    2580 flags.go:64] FLAG: --kube-reserved=""
Nov 03 21:29:11.502729 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497237    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497240    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497243    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497246    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497249    2580 flags.go:64] FLAG: --lock-file=""
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497252    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497255    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497258    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497263    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497266    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497269    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497272    2580 flags.go:64] FLAG: --logging-format="text"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497276    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497279    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497282    2580 flags.go:64] FLAG: --manifest-url=""
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497285    2580 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497290    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497293    2580 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497297    2580 flags.go:64] FLAG: --max-pods="110"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497302    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497305    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497311    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497314    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497317    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497320    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 21:29:11.503287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497323    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497330    2580 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497333    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497336    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497340    2580 flags.go:64] FLAG: --pod-cidr=""
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497343    2580 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497346    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497349    2580 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497352    2580 flags.go:64] FLAG: --port="10250"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497355    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497358    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-089416bbf08c096de"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497361    2580 flags.go:64] FLAG: --qos-reserved=""
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497364    2580 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497367    2580 flags.go:64] FLAG: --register-node="true"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497370    2580 flags.go:64] FLAG: --register-with-taints=""
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497374    2580 flags.go:64] FLAG: --registry-burst="10"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497376    2580 flags.go:64] FLAG: --registry-qps="5"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497379    2580 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497382    2580 flags.go:64] FLAG: --reserved-memory=""
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497386    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497389    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497392    2580 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497395    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497398    2580 flags.go:64] FLAG: --runonce="false"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497401    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497404    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497408    2580 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 21:29:11.503920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497411    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497415    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497419    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497422    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497426    2580 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497429    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497432    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497435    2580 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497437    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497441    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497444    2580 flags.go:64] FLAG: --system-cgroups=""
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497447    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497452    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497455    2580 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497458    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497462    2580 flags.go:64] FLAG: --tls-min-version=""
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497465    2580 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497468    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497471    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497474    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497477    2580 flags.go:64] FLAG: --v="2"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497482    2580 flags.go:64] FLAG: --version="false"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497486    2580 flags.go:64] FLAG: --vmodule=""
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497490    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497493    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 21:29:11.504562 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497666    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497671    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497675    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497678    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497681    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497683    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497687    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497690    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497694    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497697    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497700    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497703    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497706    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497710    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497714    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497717    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497719    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497723    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497725    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:11.505153 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497728    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497731    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497734    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497737    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497739    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497742    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497745    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497747    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497750    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497753    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497755    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497758    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497760    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497763    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497765    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497768    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497771    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497773    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497776    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497779    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:11.505611 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497782    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497785    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497788    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497790    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497794    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497797    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497799    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497802    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497805    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497807    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497810    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497812    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497815    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497817    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497820    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497822    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497825    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497828    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497830    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:11.506086 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497833    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497835    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497838    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497841    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497843    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497846    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497849    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497851    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497854    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497856    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497859    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497861    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497867    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497870    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497873    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497876    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497879    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497881    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497884    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:11.506583 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497887    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497889    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497892    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497894    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497897    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497899    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497902    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497905    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497908    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497910    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497913    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497916    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497918    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497921    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497923    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497926    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497928    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497931    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497933    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497936    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:11.507030 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497938    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497941    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497943    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497946    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497948    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497952    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497955    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497957    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.497960    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.497986    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498038    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498043    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498052    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498055    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498057    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498061    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:11.507492 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498063    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498066    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498069    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498071    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498074    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498076    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498079    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498082    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498084    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498087    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498090    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498093    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498095    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498098    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498101    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498103    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498105    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498108    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498111    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:11.507887 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498113    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498116    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498120    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498124    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498127    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498130    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498133    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498135    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498139    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498141    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498144    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498147    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498149    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498152    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498154    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498157    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498159    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498162    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498165    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498167    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:11.508336 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498170    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498172    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498175    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498177    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498180    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498182    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498185    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498187    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498190    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498192    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498195    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498197    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498200    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498202    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498206    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498209    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498212    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498216    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498219    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:11.508882 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498222    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498225    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498227    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498230    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498236    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498239    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498241    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498244    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498246    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498249    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498252    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498254    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498257    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498261    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498264    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498267    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498269    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498272    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498275    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:11.509328 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498277    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498280    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498283    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498286    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498288    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498291    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498293    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498296    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498299    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498302    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498304    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498309    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498311    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498314    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498317    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498320    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498323    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498325    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498329    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498332    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:11.509778 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498334    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498337    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.498339    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.504840    2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.504856    2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.504873    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.504878    2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504940    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504945    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504948    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504951    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504954    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504958    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504961    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504964    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504966    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504969    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504972    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504974    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:11.510255 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504977    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504980    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504983    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504985    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504988    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504990    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504993    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.504996    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505000    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505003    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505005    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505008    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505011    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505013    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505016    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505020    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505024    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505028    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505031    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:11.510697 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505034    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505037    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505040    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505043    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505045    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505048    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505051    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505054    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505056    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505059    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505062    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505064    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505067    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505069    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505072    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505075    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505078    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505080    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505083    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:11.511143 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505086    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505088    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505092    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505095    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505098    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505100    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505103    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505105    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505108    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505110    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505113    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505115    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505119    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505121    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505124    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505126    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505129    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505131    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505134    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505136    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:11.511599 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505139    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505141    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505144    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505146    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505148    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505151    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505153    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505156    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505159    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505161    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505164    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505167    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505170    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505173    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505176    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505178    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505181    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505184    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505187    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505189    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:11.512101 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505191    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505194    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505197    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505199    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505202    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505205    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505207    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505210    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505212    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505215    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505217    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505219    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505222    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505226    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505229    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505233    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:11.512580 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.505259    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505401    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505405    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505408    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505412    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505414    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505417    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505420    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505423    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505425    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505428    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505431    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505433    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505436    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505438    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505441    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505443    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505446    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505449    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505452    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:11.512994 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505454    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505457    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505460    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505462    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505465    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505467    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505470    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505472    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505475    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505477    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505480    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505482    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505485    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505487    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505490    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505493    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505496    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505498    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505501    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505503    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:11.513461 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505519    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505523    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505527    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505530    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505532    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505535    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505537    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505540    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505542    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505545    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505548    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505550    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505553    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505555    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505558    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505561    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505564    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505566    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505569    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505572    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:11.513971 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505574    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505577    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505579    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505582    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505585    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505589    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505592    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505595    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505598    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505600    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505603    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505606    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505609    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505611    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505616    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505619    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505623    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505626    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505629    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:11.514446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505632    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505634    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505637    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505639    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505642    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505644    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505647    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505649    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505652    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505655    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505657    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505660    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505662    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505665    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505667    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505670    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505672    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505675    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505677    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505680    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:11.514973 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505682    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505685    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505688    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505690    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505693    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505695    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505698    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505700    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.505724    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505765    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505770    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505773    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505776    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505778    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505781    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:11.515561 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505783    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505786    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505789    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505791    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505794    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505796    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505799    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505802    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505805    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505807    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505810    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505812    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505815    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505817    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505820    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505822    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505827    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505830    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505833    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505835    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:11.515918 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505838    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505841    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505843    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505846    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505850    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505854    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505856    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505859    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505862    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505864    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505867    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505870    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505872    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505875    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505879    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505882    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505885    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505888    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505890    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:11.516387 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505893    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505901    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505904    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505906    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505909    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505912    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505914    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505917    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505919    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505923    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505926    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505929    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505931    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505934    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505936    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505939    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505941    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505944    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505947    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505950    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:11.516899 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505952    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505955    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505958    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505960    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505963    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505966    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505968    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505971    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505974    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505976    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505979    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505981    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505984    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505986    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505989    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505992    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505995    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.505998    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506000    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:11.517365 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506003    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506005    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506009    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506012    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506021    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506025    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506027    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506030    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506033    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506035    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506038    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506041    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506043    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506047    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506049    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506052    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506055    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506057    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506060    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506062    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:11.517972 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506065    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:11.518446 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:11.506068    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:11.518446 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.506126    2580 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 21:29:11.518446 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.509579    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 03 21:29:11.518446 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.510854    2580 server.go:1014] "Starting client certificate rotation"
Nov 03 21:29:11.518446 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.510948    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 21:29:11.518446 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.510986    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 21:29:11.518446 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.512930    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 21:29:11.518446 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.515740    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 21:29:11.527957 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.527941    2580 log.go:25] "Validated CRI v1 runtime API"
Nov 03 21:29:11.531233 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.531212    2580 log.go:25] "Validated CRI v1 image API"
Nov 03 21:29:11.532121 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.532090    2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 21:29:11.535529 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.535490    2580 fs.go:135] Filesystem UUIDs: map[473c0aac-8d1d-4bc8-9723-fdedc9c6658f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 96804c93-4b39-43f0-ab87-d4dbce5a3177:/dev/nvme0n1p3]
Nov 03 21:29:11.535583 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.535530    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 03 21:29:11.541154 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.541045    2580 manager.go:217] Machine: {Timestamp:2026-11-03 21:29:11.539602249 +0000 UTC m=+0.349537678 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3076598 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec246965302b872c5175edbeb1aa6489 SystemUUID:ec246965-302b-872c-5175-edbeb1aa6489 BootID:6df89aeb-1239-4247-9deb-8bea5ff07a7b Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c9:10:1d:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c9:10:1d:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:f5:d9:c1:8a:bb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 21:29:11.541154 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.541152    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 21:29:11.541272 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.541257    2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 21:29:11.544084 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.544058    2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 21:29:11.544243 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.544086    2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-212.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 21:29:11.544794 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.544781    2580 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 21:29:11.544849 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.544799    2580 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 21:29:11.544849 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.544841    2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 21:29:11.544922 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.544860    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 21:29:11.545631 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.545617    2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 21:29:11.549262 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.549247    2580 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 21:29:11.549388 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.549378    2580 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 21:29:11.550095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.550084    2580 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 21:29:11.550150 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.550102    2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 21:29:11.550150 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.550117    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 21:29:11.550150 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.550131    2580 kubelet.go:405] "Adding apiserver pod source"
Nov 03 21:29:11.550150 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.550142    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 21:29:11.550481 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.550468    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 21:29:11.552812 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.552793    2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 21:29:11.557282 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.557268    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 21:29:11.558111 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558098    2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 21:29:11.558149 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558130    2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 21:29:11.558295 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558282    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 21:29:11.558333 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558306    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 21:29:11.558333 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558315    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 21:29:11.558382 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558359    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 21:29:11.558382 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558373    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 21:29:11.558382 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558381    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 21:29:11.558954 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558944    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 21:29:11.558985 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558956    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 21:29:11.558985 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558968    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 21:29:11.558985 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558974    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 21:29:11.559064 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558987    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 21:29:11.559064 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.558996    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 21:29:11.559467 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.559457    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 21:29:11.559467 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.559467    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 21:29:11.563221 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.563208    2580 server.go:1263] "Started kubelet"
Nov 03 21:29:11.563357 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.563290    2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 21:29:11.567696 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.563380    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 21:29:11.567779 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.567733    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 21:29:11.567810 ip-10-0-137-212 systemd[1]: Started Kubernetes Kubelet.
Nov 03 21:29:11.568831 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.568753    2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 21:29:11.571065 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.571035    2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 03 21:29:11.572580 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.572567    2580 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 21:29:11.574878 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.574859    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 21:29:11.575439 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.575329    2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 21:29:11.576189 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.576159    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:11.576256 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.576188    2580 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 21:29:11.576256 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.576205    2580 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 21:29:11.576314 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.576285    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 21:29:11.576397 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.576386    2580 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 21:29:11.576432 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.576397    2580 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 21:29:11.576882 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.576863    2580 factory.go:55] Registering systemd factory
Nov 03 21:29:11.576965 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.576888    2580 factory.go:223] Registration of the systemd container factory successfully
Nov 03 21:29:11.577172 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.577140    2580 factory.go:153] Registering CRI-O factory
Nov 03 21:29:11.577172 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.577159    2580 factory.go:223] Registration of the crio container factory successfully
Nov 03 21:29:11.577308 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.577209    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 21:29:11.577308 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.577224    2580 factory.go:103] Registering Raw factory
Nov 03 21:29:11.577308 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.577234    2580 manager.go:1199] Started watching for new ooms in manager
Nov 03 21:29:11.578129 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.578114    2580 manager.go:319] Starting recovery of all containers
Nov 03 21:29:11.587457 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.587439    2580 manager.go:324] Recovery completed
Nov 03 21:29:11.588808 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.588783    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 21:29:11.588917 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.588846    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ncc5j"
Nov 03 21:29:11.591886 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.591862    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ncc5j"
Nov 03 21:29:11.593039 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.593024    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 21:29:11.595194 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.595178    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientMemory"
Nov 03 21:29:11.595310 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.595300    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasNoDiskPressure"
Nov 03 21:29:11.595390 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.595382    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientPID"
Nov 03 21:29:11.596833 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.596811    2580 cpu_manager.go:226] "Starting" policy="none"
Nov 03 21:29:11.596907 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.596842    2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 21:29:11.596907 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.596828    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 21:29:11.596907 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.596862    2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 21:29:11.597752 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.597725    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientMemory"
Nov 03 21:29:11.597867 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.597764    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasNoDiskPressure"
Nov 03 21:29:11.597867 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.597779    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientPID"
Nov 03 21:29:11.600597 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.600571    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-212.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 03 21:29:11.600597 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.600576    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-212.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 21:29:11.601568 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.600582    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-212.ec2.internal.18e4a6416eda31fe  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-212.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-212.ec2.internal,},FirstTimestamp:2026-11-03 21:29:11.563178494 +0000 UTC m=+0.373113920,LastTimestamp:2026-11-03 21:29:11.563178494 +0000 UTC m=+0.373113920,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-212.ec2.internal,}"
Nov 03 21:29:11.616123 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.615981    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-212.ec2.internal.18e4a64170c3fec5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-212.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-212.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-212.ec2.internal,},FirstTimestamp:2026-11-03 21:29:11.595278021 +0000 UTC m=+0.405213454,LastTimestamp:2026-11-03 21:29:11.595278021 +0000 UTC m=+0.405213454,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-212.ec2.internal,}"
Nov 03 21:29:11.616123 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.616051    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 03 21:29:11.616310 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.616042    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 03 21:29:11.642182 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.642103    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-212.ec2.internal.18e4a64170c568d1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-212.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-212.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-212.ec2.internal,},FirstTimestamp:2026-11-03 21:29:11.595370705 +0000 UTC m=+0.405306133,LastTimestamp:2026-11-03 21:29:11.595370705 +0000 UTC m=+0.405306133,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-212.ec2.internal,}"
Nov 03 21:29:11.642182 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.642165    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-212.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 03 21:29:11.656522 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.656449    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-212.ec2.internal.18e4a64170c6d9b2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-212.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-212.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-212.ec2.internal,},FirstTimestamp:2026-11-03 21:29:11.595465138 +0000 UTC m=+0.405400569,LastTimestamp:2026-11-03 21:29:11.595465138 +0000 UTC m=+0.405400569,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-212.ec2.internal,}"
Nov 03 21:29:11.677265 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.677247    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:11.692310 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.692243    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-212.ec2.internal.18e4a64170c3fec5\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-212.ec2.internal.18e4a64170c3fec5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-212.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-212.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-212.ec2.internal,},FirstTimestamp:2026-11-03 21:29:11.595278021 +0000 UTC m=+0.405213454,LastTimestamp:2026-11-03 21:29:11.597752205 +0000 UTC m=+0.407687636,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-212.ec2.internal,}"
Nov 03 21:29:11.728052 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.728033    2580 policy_none.go:50] "Start"
Nov 03 21:29:11.728052 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.728051    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 21:29:11.728168 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.728063    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 21:29:11.732780 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.732704    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-212.ec2.internal.18e4a64170c568d1\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-212.ec2.internal.18e4a64170c568d1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-212.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-212.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-212.ec2.internal,},FirstTimestamp:2026-11-03 21:29:11.595370705 +0000 UTC m=+0.405306133,LastTimestamp:2026-11-03 21:29:11.597770674 +0000 UTC m=+0.407706101,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-212.ec2.internal,}"
Nov 03 21:29:11.733721 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.733707    2580 policy_none.go:44] "Start"
Nov 03 21:29:11.779742 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.766108    2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 21:29:11.779742 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.768546    2580 manager.go:341] "Starting Device Plugin manager"
Nov 03 21:29:11.779742 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.768975    2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 21:29:11.779742 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.768986    2580 server.go:87] "Starting device plugin registration server"
Nov 03 21:29:11.779742 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.769187    2580 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 21:29:11.779742 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.769198    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 21:29:11.779742 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.769303    2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 21:29:11.779742 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.769390    2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 21:29:11.779742 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.769400    2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 21:29:11.779742 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.769885    2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 21:29:11.779742 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.769926    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:11.827030 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.827003    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 21:29:11.828152 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.828134    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 21:29:11.828152 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.828152    2580 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 21:29:11.828249 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.828168    2580 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 21:29:11.828249 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.828196    2580 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 03 21:29:11.829247 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.829230    2580 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-212.ec2.internal\" not found" node="ip-10-0-137-212.ec2.internal"
Nov 03 21:29:11.842761 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.842740    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 21:29:11.869850 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.869833    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 21:29:11.870743 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.870729    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientMemory"
Nov 03 21:29:11.870803 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.870758    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasNoDiskPressure"
Nov 03 21:29:11.870803 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.870768    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientPID"
Nov 03 21:29:11.870803 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.870798    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-137-212.ec2.internal"
Nov 03 21:29:11.890080 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.890063    2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-137-212.ec2.internal"
Nov 03 21:29:11.890131 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.890085    2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-212.ec2.internal\": node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:11.928816 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.928794    2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"]
Nov 03 21:29:11.928892 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.928848    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 21:29:11.929751 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.929739    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientMemory"
Nov 03 21:29:11.929811 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.929766    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasNoDiskPressure"
Nov 03 21:29:11.929811 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.929776    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientPID"
Nov 03 21:29:11.931759 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.931748    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 21:29:11.931928 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.931915    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:11.931976 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.931946    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 21:29:11.932477 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.932463    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientMemory"
Nov 03 21:29:11.932554 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.932486    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientMemory"
Nov 03 21:29:11.932554 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.932491    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasNoDiskPressure"
Nov 03 21:29:11.932554 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.932521    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasNoDiskPressure"
Nov 03 21:29:11.932554 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.932527    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientPID"
Nov 03 21:29:11.932554 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.932531    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientPID"
Nov 03 21:29:11.934644 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.934629    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:11.934727 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.934653    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 21:29:11.935305 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.935291    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientMemory"
Nov 03 21:29:11.935391 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.935317    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasNoDiskPressure"
Nov 03 21:29:11.935391 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.935330    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientPID"
Nov 03 21:29:11.949480 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.949460    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:11.954359 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.954341    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-212.ec2.internal\" not found" node="ip-10-0-137-212.ec2.internal"
Nov 03 21:29:11.956207 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:11.956194    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-212.ec2.internal\" not found" node="ip-10-0-137-212.ec2.internal"
Nov 03 21:29:11.978827 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.978808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9caeb2eb053e15a50e91a5dd1ebbc6a1-config\") pod \"kube-apiserver-proxy-ip-10-0-137-212.ec2.internal\" (UID: \"9caeb2eb053e15a50e91a5dd1ebbc6a1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:11.978890 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.978832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/86212ec82a64966b4f712c3fdb2b1cf2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" (UID: \"86212ec82a64966b4f712c3fdb2b1cf2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:11.978890 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:11.978850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86212ec82a64966b4f712c3fdb2b1cf2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" (UID: \"86212ec82a64966b4f712c3fdb2b1cf2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:12.050044 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:12.049986    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:12.079554 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.079525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/86212ec82a64966b4f712c3fdb2b1cf2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" (UID: \"86212ec82a64966b4f712c3fdb2b1cf2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:12.079624 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.079557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86212ec82a64966b4f712c3fdb2b1cf2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" (UID: \"86212ec82a64966b4f712c3fdb2b1cf2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:12.079624 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.079573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9caeb2eb053e15a50e91a5dd1ebbc6a1-config\") pod \"kube-apiserver-proxy-ip-10-0-137-212.ec2.internal\" (UID: \"9caeb2eb053e15a50e91a5dd1ebbc6a1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:12.079691 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.079632    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/86212ec82a64966b4f712c3fdb2b1cf2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" (UID: \"86212ec82a64966b4f712c3fdb2b1cf2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:12.079691 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.079640    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86212ec82a64966b4f712c3fdb2b1cf2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" (UID: \"86212ec82a64966b4f712c3fdb2b1cf2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:12.079691 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.079633    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9caeb2eb053e15a50e91a5dd1ebbc6a1-config\") pod \"kube-apiserver-proxy-ip-10-0-137-212.ec2.internal\" (UID: \"9caeb2eb053e15a50e91a5dd1ebbc6a1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:12.150677 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:12.150655    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:12.251467 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:12.251441    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:12.258073 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.258048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:12.259222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.259208    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:12.354921 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:12.354696    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:12.454664 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.454640    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 21:29:12.455729 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:12.455711    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:12.510867 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.510830    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 03 21:29:12.511325 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.511004    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 03 21:29:12.511325 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.511006    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 03 21:29:12.556502 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:12.556478    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:12.657635 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:12.657565    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:12.730196 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.730159    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 21:24:11 +0000 UTC" deadline="2028-04-03 17:17:48.810952738 +0000 UTC"
Nov 03 21:29:12.730196 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.730191    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12403h48m36.080764513s"
Nov 03 21:29:12.751972 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.751948    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 21:29:12.753355 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.753325    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientMemory"
Nov 03 21:29:12.753462 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.753377    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasNoDiskPressure"
Nov 03 21:29:12.753462 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.753398    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientPID"
Nov 03 21:29:12.758671 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:12.758649    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:12.858765 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:12.858734    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:12.861053 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:12.861004    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9caeb2eb053e15a50e91a5dd1ebbc6a1.slice/crio-c5320a9e6eb91b05defae8ce80631832933ef7a7280f7ede24826d59c4f0e0b6 WatchSource:0}: Error finding container c5320a9e6eb91b05defae8ce80631832933ef7a7280f7ede24826d59c4f0e0b6: Status 404 returned error can't find the container with id c5320a9e6eb91b05defae8ce80631832933ef7a7280f7ede24826d59c4f0e0b6
Nov 03 21:29:12.869774 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:12.869751    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:29:12.869855 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:12.869824    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86212ec82a64966b4f712c3fdb2b1cf2.slice/crio-454db50f9c3353a5d5e1ab583eb65f3ef2196c37ce2190dc7677978a33323636 WatchSource:0}: Error finding container 454db50f9c3353a5d5e1ab583eb65f3ef2196c37ce2190dc7677978a33323636: Status 404 returned error can't find the container with id 454db50f9c3353a5d5e1ab583eb65f3ef2196c37ce2190dc7677978a33323636
Nov 03 21:29:12.959896 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:12.959823    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:13.046807 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.046782    2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 21:29:13.076591 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.076563    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:13.101138 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.101116    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 21:29:13.103360 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.103342    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:13.125197 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.125176    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 21:29:13.205791 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.205762    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 21:29:13.551684 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.551650    2580 apiserver.go:52] "Watching apiserver"
Nov 03 21:29:13.576500 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.576471    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 03 21:29:13.583564 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.583522    2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 21:29:13.583983 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.583957    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-529hb","openshift-ovn-kubernetes/ovnkube-node-6xdtq","openshift-cluster-node-tuning-operator/tuned-x55l2","openshift-multus/multus-additional-cni-plugins-swgh5","openshift-multus/multus-q2zcj","kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal","openshift-image-registry/node-ca-wjgz5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal","openshift-network-diagnostics/network-check-target-7s62j","kube-system/global-pull-secret-syncer-vmr9k","openshift-multus/network-metrics-daemon-4gsg9","openshift-network-operator/iptables-alerter-wk2lw","kube-system/konnectivity-agent-79tpv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"]
Nov 03 21:29:13.586590 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.586558    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:13.588874 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.588848    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:13.592193 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.592170    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.594462 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.594441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.597339 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.597319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.599561 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.599536    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.602352 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.602335    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:13.602462 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:13.602419    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:13.602963 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.602870    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6tvtx\""
Nov 03 21:29:13.602963 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.602927    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 21:29:13.602963 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.602935    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 21:29:13.604843 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.604824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:13.606568 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.606451    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 21:29:13.606667 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.606635    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 21:29:13.606851 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.606829    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-xgwf4\""
Nov 03 21:29:13.606925 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.606908    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 21:29:13.607447 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.607426    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.607772 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.607750    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 21:29:13.607967 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.607953    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tnnmd\""
Nov 03 21:29:13.609932 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.609875    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 21:29:13.610044 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.610013    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 21:29:13.612267 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.610374    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:13.612267 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:13.610457    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:13.613619 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.613591    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:13.613751 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:13.613663    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:13.615119 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.615095    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-d5mwz\""
Nov 03 21:29:13.615224 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.615129    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 21:29:13.615224 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.615156    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 21:29:13.615224 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.615217    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 21:29:13.615394 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.615283    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 21:29:13.615394 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.615299    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 21:29:13.615394 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.615376    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-6tkv6\""
Nov 03 21:29:13.615540 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.615404    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 21:29:13.616485 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.616464    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 21:29:13.616562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.616483    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 21:29:13.616647 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.616593    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w4krc\""
Nov 03 21:29:13.617093 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.617075    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:13.621339 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.621318    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vzdjb\""
Nov 03 21:29:13.628562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.628541    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 21:29:13.628665 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.628577    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6xptx"
Nov 03 21:29:13.678319 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.678298    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 21:29:13.686310 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.686286    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 21:29:13.687442 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687422    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovnkube-script-lib\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.687533 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlk69\" (UniqueName: \"kubernetes.io/projected/af4170fc-093a-4066-bf79-4f57388f073d-kube-api-access-jlk69\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:13.687533 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysctl-conf\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.687673 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tfpz\" (UniqueName: \"kubernetes.io/projected/47bb3d82-52e3-4e39-8b1a-1deb886a29db-kube-api-access-7tfpz\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.687673 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-cnibin\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.687673 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-netns\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.687832 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-cni-bin\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.687832 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-log-socket\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.687832 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-etc-selinux\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.687832 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-hosts-file\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:13.687832 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687795    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.687832 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b978b73d-666a-4c25-b959-74d3e14c7bbf-dbus\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:13.688134 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687851    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a287e20-e2d0-451a-97ce-b4fddffeb788-serviceca\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:13.688134 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cnibin\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.688134 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-system-cni-dir\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.688134 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cni-binary-copy\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.688134 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687958    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-cni-multus\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.688134 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.687983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-ovn\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.688134 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-node-log\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.688134 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.688134 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:13.688134 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-sys\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47bb3d82-52e3-4e39-8b1a-1deb886a29db-tmp\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-daemon-config\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-etc-kubernetes\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b978b73d-666a-4c25-b959-74d3e14c7bbf-kubelet-config\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a287e20-e2d0-451a-97ce-b4fddffeb788-host\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-modprobe-d\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-run\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-lib-modules\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-cni-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-socket-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688442    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fh5b\" (UniqueName: \"kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b\") pod \"network-check-target-7s62j\" (UID: \"29ad1278-e154-41fb-8648-0adc8d7a648f\") " pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-var-lib-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-cni-bin\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688522    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af4170fc-093a-4066-bf79-4f57388f073d-iptables-alerter-script\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:13.688660 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688580    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-cni-netd\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bcb1ef0-aa94-4932-99a6-53178e770a68-cni-binary-copy\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688682    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-run-netns\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkpsd\" (UniqueName: \"kubernetes.io/projected/3a287e20-e2d0-451a-97ce-b4fddffeb788-kube-api-access-gkpsd\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysconfig\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-kubernetes\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-host\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq4wr\" (UniqueName: \"kubernetes.io/projected/b7e2c320-fbf4-40e6-938d-b4e1377d769c-kube-api-access-dq4wr\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688868    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lz9n\" (UniqueName: \"kubernetes.io/projected/aa5882bd-1e86-428d-afec-490ebd2da878-kube-api-access-6lz9n\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688896    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-systemd-units\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovn-node-metrics-cert\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/45263f32-066f-4bdf-84b9-a3ebd1f33f90-agent-certs\") pod \"konnectivity-agent-79tpv\" (UID: \"45263f32-066f-4bdf-84b9-a3ebd1f33f90\") " pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688974    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-tmp-dir\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.688999    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gfqr\" (UniqueName: \"kubernetes.io/projected/fe26d35e-47a4-497a-9fe1-64cc807267e9-kube-api-access-2gfqr\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.689461 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/45263f32-066f-4bdf-84b9-a3ebd1f33f90-konnectivity-ca\") pod \"konnectivity-agent-79tpv\" (UID: \"45263f32-066f-4bdf-84b9-a3ebd1f33f90\") " pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-system-cni-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689126    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-multus-certs\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-registration-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-sys-fs\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689210    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-tuned\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvfs2\" (UniqueName: \"kubernetes.io/projected/8bcb1ef0-aa94-4932-99a6-53178e770a68-kube-api-access-bvfs2\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-etc-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysctl-d\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-kubelet\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689341    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-slash\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-env-overrides\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689397    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-device-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82wfl\" (UniqueName: \"kubernetes.io/projected/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-kube-api-access-82wfl\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-kubelet\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.690261 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af4170fc-093a-4066-bf79-4f57388f073d-host-slash\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:13.691095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-var-lib-kubelet\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.691095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-socket-dir-parent\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.691095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-conf-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.691095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-systemd\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.691095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689651    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.691095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689680    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:13.691095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx8fk\" (UniqueName: \"kubernetes.io/projected/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-kube-api-access-dx8fk\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:13.691095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-systemd\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.691095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-k8s-cni-cncf-io\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.691095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-hostroot\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.691095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.689903    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovnkube-config\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.711478 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.711459    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 21:29:13.711625 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.711550    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 21:29:13.711625 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.711608    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 21:29:13.711916 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.711877    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 21:29:13.711983 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.711941    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 21:29:13.712039 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.712004    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 21:29:13.712095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.712046    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 21:29:13.712095 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.712085    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 21:29:13.712195 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.712139    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 21:29:13.712257 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.712209    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mhg7x\""
Nov 03 21:29:13.712312 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.712288    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 21:29:13.712414 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.712383    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4tzc5\""
Nov 03 21:29:13.712547 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.712486    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 21:29:13.790302 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-socket-dir-parent\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.790488 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-conf-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.790488 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-systemd\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.790488 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.790488 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:13.790488 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx8fk\" (UniqueName: \"kubernetes.io/projected/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-kube-api-access-dx8fk\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:13.790488 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-systemd\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.790488 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-k8s-cni-cncf-io\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-hostroot\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790528    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovnkube-config\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovnkube-script-lib\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790606    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlk69\" (UniqueName: \"kubernetes.io/projected/af4170fc-093a-4066-bf79-4f57388f073d-kube-api-access-jlk69\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysctl-conf\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tfpz\" (UniqueName: \"kubernetes.io/projected/47bb3d82-52e3-4e39-8b1a-1deb886a29db-kube-api-access-7tfpz\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:13.790701    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:13.790798    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret podName:b978b73d-666a-4c25-b959-74d3e14c7bbf nodeName:}" failed. No retries permitted until 2026-11-03 21:29:14.290767156 +0000 UTC m=+3.100702589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret") pod "global-pull-secret-syncer-vmr9k" (UID: "b978b73d-666a-4c25-b959-74d3e14c7bbf") : object "kube-system"/"original-pull-secret" not registered
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790822    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-cnibin\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791127    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-socket-dir-parent\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791142    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-systemd\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791180    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-conf-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791190    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-k8s-cni-cncf-io\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791232    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-hostroot\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791255    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-systemd\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.790700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-cnibin\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-netns\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.795775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-cni-bin\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-log-socket\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-etc-selinux\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-hosts-file\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b978b73d-666a-4c25-b959-74d3e14c7bbf-dbus\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a287e20-e2d0-451a-97ce-b4fddffeb788-serviceca\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cnibin\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-system-cni-dir\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cni-binary-copy\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-cni-multus\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-ovn\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791733    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-node-log\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-sys\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47bb3d82-52e3-4e39-8b1a-1deb886a29db-tmp\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.796617 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-daemon-config\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-etc-kubernetes\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791976    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovnkube-config\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.791978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b978b73d-666a-4c25-b959-74d3e14c7bbf-kubelet-config\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792038    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b978b73d-666a-4c25-b959-74d3e14c7bbf-kubelet-config\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a287e20-e2d0-451a-97ce-b4fddffeb788-host\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-modprobe-d\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792093    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-netns\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-run\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792148    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-cni-bin\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-lib-modules\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792196    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-log-socket\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-cni-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-socket-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fh5b\" (UniqueName: \"kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b\") pod \"network-check-target-7s62j\" (UID: \"29ad1278-e154-41fb-8648-0adc8d7a648f\") " pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-var-lib-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-cni-bin\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.797406 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af4170fc-093a-4066-bf79-4f57388f073d-iptables-alerter-script\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792417    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-etc-selinux\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-cni-netd\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792488    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-hosts-file\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792524    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bcb1ef0-aa94-4932-99a6-53178e770a68-cni-binary-copy\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-run-netns\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkpsd\" (UniqueName: \"kubernetes.io/projected/3a287e20-e2d0-451a-97ce-b4fddffeb788-kube-api-access-gkpsd\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysconfig\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-kubernetes\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-host\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq4wr\" (UniqueName: \"kubernetes.io/projected/b7e2c320-fbf4-40e6-938d-b4e1377d769c-kube-api-access-dq4wr\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792809    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lz9n\" (UniqueName: \"kubernetes.io/projected/aa5882bd-1e86-428d-afec-490ebd2da878-kube-api-access-6lz9n\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-systemd-units\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovn-node-metrics-cert\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.798222 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.792917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/45263f32-066f-4bdf-84b9-a3ebd1f33f90-agent-certs\") pod \"konnectivity-agent-79tpv\" (UID: \"45263f32-066f-4bdf-84b9-a3ebd1f33f90\") " pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793127    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-tmp-dir\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gfqr\" (UniqueName: \"kubernetes.io/projected/fe26d35e-47a4-497a-9fe1-64cc807267e9-kube-api-access-2gfqr\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/45263f32-066f-4bdf-84b9-a3ebd1f33f90-konnectivity-ca\") pod \"konnectivity-agent-79tpv\" (UID: \"45263f32-066f-4bdf-84b9-a3ebd1f33f90\") " pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-system-cni-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793622    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-multus-certs\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-registration-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-sys-fs\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-tuned\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvfs2\" (UniqueName: \"kubernetes.io/projected/8bcb1ef0-aa94-4932-99a6-53178e770a68-kube-api-access-bvfs2\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793766    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovnkube-script-lib\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-etc-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793935    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b978b73d-666a-4c25-b959-74d3e14c7bbf-dbus\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.793974    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.794132    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysctl-conf\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.799018 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.794421    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a287e20-e2d0-451a-97ce-b4fddffeb788-serviceca\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.794426    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.794467    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a287e20-e2d0-451a-97ce-b4fddffeb788-host\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.794489    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cnibin\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.794546    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-system-cni-dir\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.794608    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-socket-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.794882    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-var-lib-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.794929    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-cni-bin\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.795038    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cni-binary-copy\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.795708    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-cni-multus\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.795853    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.795964    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-ovn\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.796007    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.796024    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-run-netns\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.796049    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-node-log\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.796246    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-tmp-dir\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.796313    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysconfig\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.796403    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-kubernetes\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.799810 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.796455    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-host\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.796501    2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.797089    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-systemd-units\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.797178    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-multus-certs\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.797238    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bcb1ef0-aa94-4932-99a6-53178e770a68-cni-binary-copy\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.797393    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:13.797448    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:13.797501    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs podName:ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a nodeName:}" failed. No retries permitted until 2026-11-03 21:29:14.297477859 +0000 UTC m=+3.107413289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs") pod "network-metrics-daemon-4gsg9" (UID: "ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.797530    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-system-cni-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.797603    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-sys\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.797642    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-sys-fs\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.797699    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/45263f32-066f-4bdf-84b9-a3ebd1f33f90-konnectivity-ca\") pod \"konnectivity-agent-79tpv\" (UID: \"45263f32-066f-4bdf-84b9-a3ebd1f33f90\") " pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.797764    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-etc-kubernetes\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.797775    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-registration-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.797869    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-cni-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysctl-d\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-kubelet\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-slash\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.800702 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-env-overrides\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-device-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798421    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-modprobe-d\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798474    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-run\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798570    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-lib-modules\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82wfl\" (UniqueName: \"kubernetes.io/projected/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-kube-api-access-82wfl\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-kubelet\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af4170fc-093a-4066-bf79-4f57388f073d-host-slash\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-var-lib-kubelet\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798852    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-var-lib-kubelet\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798901    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-device-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.798952    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-cni-netd\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.799347    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-daemon-config\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.799702    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af4170fc-093a-4066-bf79-4f57388f073d-iptables-alerter-script\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.800069    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-env-overrides\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.800164    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysctl-d\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.800212    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-kubelet\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.800252    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-slash\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.801666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.800296    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-kubelet\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.802567 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.800535    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-etc-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.802567 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.800601    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af4170fc-093a-4066-bf79-4f57388f073d-host-slash\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:13.802567 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.800662    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/45263f32-066f-4bdf-84b9-a3ebd1f33f90-agent-certs\") pod \"konnectivity-agent-79tpv\" (UID: \"45263f32-066f-4bdf-84b9-a3ebd1f33f90\") " pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:13.802567 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.797353    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.802567 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.801858    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovn-node-metrics-cert\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.802567 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.801937    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47bb3d82-52e3-4e39-8b1a-1deb886a29db-tmp\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.805014 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.804989    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-tuned\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.812969 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:13.812711    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 21:29:13.812969 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:13.812742    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 21:29:13.812969 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:13.812761    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2fh5b for pod openshift-network-diagnostics/network-check-target-7s62j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 21:29:13.812969 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:13.812827    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b podName:29ad1278-e154-41fb-8648-0adc8d7a648f nodeName:}" failed. No retries permitted until 2026-11-03 21:29:14.312807322 +0000 UTC m=+3.122742761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2fh5b" (UniqueName: "kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b") pod "network-check-target-7s62j" (UID: "29ad1278-e154-41fb-8648-0adc8d7a648f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 21:29:13.815875 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.815788    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tfpz\" (UniqueName: \"kubernetes.io/projected/47bb3d82-52e3-4e39-8b1a-1deb886a29db-kube-api-access-7tfpz\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.816338 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.816315    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlk69\" (UniqueName: \"kubernetes.io/projected/af4170fc-093a-4066-bf79-4f57388f073d-kube-api-access-jlk69\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:13.818330 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.818303    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq4wr\" (UniqueName: \"kubernetes.io/projected/b7e2c320-fbf4-40e6-938d-b4e1377d769c-kube-api-access-dq4wr\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.818805 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.818781    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gfqr\" (UniqueName: \"kubernetes.io/projected/fe26d35e-47a4-497a-9fe1-64cc807267e9-kube-api-access-2gfqr\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.819516 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.819490    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkpsd\" (UniqueName: \"kubernetes.io/projected/3a287e20-e2d0-451a-97ce-b4fddffeb788-kube-api-access-gkpsd\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:13.820038 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.820013    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lz9n\" (UniqueName: \"kubernetes.io/projected/aa5882bd-1e86-428d-afec-490ebd2da878-kube-api-access-6lz9n\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.820401 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.820375    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82wfl\" (UniqueName: \"kubernetes.io/projected/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-kube-api-access-82wfl\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:13.821090 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.821064    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvfs2\" (UniqueName: \"kubernetes.io/projected/8bcb1ef0-aa94-4932-99a6-53178e770a68-kube-api-access-bvfs2\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.821696 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.821674    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx8fk\" (UniqueName: \"kubernetes.io/projected/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-kube-api-access-dx8fk\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:13.833559 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.833502    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal" event={"ID":"86212ec82a64966b4f712c3fdb2b1cf2","Type":"ContainerStarted","Data":"454db50f9c3353a5d5e1ab583eb65f3ef2196c37ce2190dc7677978a33323636"}
Nov 03 21:29:13.834835 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.834809    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal" event={"ID":"9caeb2eb053e15a50e91a5dd1ebbc6a1","Type":"ContainerStarted","Data":"c5320a9e6eb91b05defae8ce80631832933ef7a7280f7ede24826d59c4f0e0b6"}
Nov 03 21:29:13.901801 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.901763    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:13.913538 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:13.913498    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a287e20_e2d0_451a_97ce_b4fddffeb788.slice/crio-03f2395f5e0075fcc939aa1646b0cb4da91a99c2815ed1b182b4ce31dd7107d0 WatchSource:0}: Error finding container 03f2395f5e0075fcc939aa1646b0cb4da91a99c2815ed1b182b4ce31dd7107d0: Status 404 returned error can't find the container with id 03f2395f5e0075fcc939aa1646b0cb4da91a99c2815ed1b182b4ce31dd7107d0
Nov 03 21:29:13.916397 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.916354    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:13.921707 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.921686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:13.932476 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:13.932444    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47bb3d82_52e3_4e39_8b1a_1deb886a29db.slice/crio-1754f54422e37a4e203375cd9ba0dc715ee426d9647fb668760b3e0299c3b742 WatchSource:0}: Error finding container 1754f54422e37a4e203375cd9ba0dc715ee426d9647fb668760b3e0299c3b742: Status 404 returned error can't find the container with id 1754f54422e37a4e203375cd9ba0dc715ee426d9647fb668760b3e0299c3b742
Nov 03 21:29:13.943293 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.943272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:13.947330 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.947306    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:13.954173 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:13.954150    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bcb1ef0_aa94_4932_99a6_53178e770a68.slice/crio-dd13bc15ca19c42bc46373cc3e0da80e0b5d90e2184c748cfea0b9bf9c6c2938 WatchSource:0}: Error finding container dd13bc15ca19c42bc46373cc3e0da80e0b5d90e2184c748cfea0b9bf9c6c2938: Status 404 returned error can't find the container with id dd13bc15ca19c42bc46373cc3e0da80e0b5d90e2184c748cfea0b9bf9c6c2938
Nov 03 21:29:13.955128 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.955105    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:13.956917 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.956902    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:13.960089 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:13.960067    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e2c320_fbf4_40e6_938d_b4e1377d769c.slice/crio-7cd88de0cca83cd280cedb54ec74f8fc1e88469fbb76f7999c8e8d74bc84bdce WatchSource:0}: Error finding container 7cd88de0cca83cd280cedb54ec74f8fc1e88469fbb76f7999c8e8d74bc84bdce: Status 404 returned error can't find the container with id 7cd88de0cca83cd280cedb54ec74f8fc1e88469fbb76f7999c8e8d74bc84bdce
Nov 03 21:29:13.971709 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:13.971679    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa5882bd_1e86_428d_afec_490ebd2da878.slice/crio-b3ea48873e2ce64601b2cb0af1686e3d7a0231bb0c3f3a61a6c5a05947a9fddf WatchSource:0}: Error finding container b3ea48873e2ce64601b2cb0af1686e3d7a0231bb0c3f3a61a6c5a05947a9fddf: Status 404 returned error can't find the container with id b3ea48873e2ce64601b2cb0af1686e3d7a0231bb0c3f3a61a6c5a05947a9fddf
Nov 03 21:29:13.975204 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:13.975099    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe02b31_bc0e_4ffc_916d_f6cd60b68a70.slice/crio-155e9bae8c395f1c283b407f9e61547fa8dfd5178e22a3ad8c814f28fa8061c5 WatchSource:0}: Error finding container 155e9bae8c395f1c283b407f9e61547fa8dfd5178e22a3ad8c814f28fa8061c5: Status 404 returned error can't find the container with id 155e9bae8c395f1c283b407f9e61547fa8dfd5178e22a3ad8c814f28fa8061c5
Nov 03 21:29:13.977059 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.977026    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:13.984907 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:13.984727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:13.995422 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:13.995398    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe26d35e_47a4_497a_9fe1_64cc807267e9.slice/crio-e92bf21eb5fcd6539d298e25967b983601226c1e20485c9dbc9201a88fa37f8f WatchSource:0}: Error finding container e92bf21eb5fcd6539d298e25967b983601226c1e20485c9dbc9201a88fa37f8f: Status 404 returned error can't find the container with id e92bf21eb5fcd6539d298e25967b983601226c1e20485c9dbc9201a88fa37f8f
Nov 03 21:29:14.302997 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.302960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:14.303146 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.303031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:14.303146 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:14.303093    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 21:29:14.303222 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:14.303173    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret podName:b978b73d-666a-4c25-b959-74d3e14c7bbf nodeName:}" failed. No retries permitted until 2026-11-03 21:29:15.3031577 +0000 UTC m=+4.113093130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret") pod "global-pull-secret-syncer-vmr9k" (UID: "b978b73d-666a-4c25-b959-74d3e14c7bbf") : object "kube-system"/"original-pull-secret" not registered
Nov 03 21:29:14.303222 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:14.303191    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 21:29:14.303290 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:14.303259    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs podName:ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a nodeName:}" failed. No retries permitted until 2026-11-03 21:29:15.303230281 +0000 UTC m=+4.113165709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs") pod "network-metrics-daemon-4gsg9" (UID: "ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 21:29:14.404465 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.404424    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fh5b\" (UniqueName: \"kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b\") pod \"network-check-target-7s62j\" (UID: \"29ad1278-e154-41fb-8648-0adc8d7a648f\") " pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:14.404653 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:14.404592    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 21:29:14.404653 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:14.404616    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 21:29:14.404653 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:14.404629    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2fh5b for pod openshift-network-diagnostics/network-check-target-7s62j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 21:29:14.404818 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:14.404684    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b podName:29ad1278-e154-41fb-8648-0adc8d7a648f nodeName:}" failed. No retries permitted until 2026-11-03 21:29:15.404664568 +0000 UTC m=+4.214600004 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2fh5b" (UniqueName: "kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b") pod "network-check-target-7s62j" (UID: "29ad1278-e154-41fb-8648-0adc8d7a648f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 21:29:14.463244 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.463172    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 21:29:14.632891 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.632724    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 21:24:13 +0000 UTC" deadline="2028-07-24 15:54:10.262687108 +0000 UTC"
Nov 03 21:29:14.632891 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.632781    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15090h24m55.629910275s"
Nov 03 21:29:14.829322 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.829288    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:14.829524 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:14.829447    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:14.829599 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.829574    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:14.829676 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:14.829655    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:14.854297 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.854264    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q2zcj" event={"ID":"8bcb1ef0-aa94-4932-99a6-53178e770a68","Type":"ContainerStarted","Data":"dd13bc15ca19c42bc46373cc3e0da80e0b5d90e2184c748cfea0b9bf9c6c2938"}
Nov 03 21:29:14.866254 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.866222    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-79tpv" event={"ID":"45263f32-066f-4bdf-84b9-a3ebd1f33f90","Type":"ContainerStarted","Data":"6881e91a5d0f2a64c8329625738daf8aa57fa67891e0c2f31e3afb9d3aca3e73"}
Nov 03 21:29:14.874333 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.874306    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq" event={"ID":"fe26d35e-47a4-497a-9fe1-64cc807267e9","Type":"ContainerStarted","Data":"e92bf21eb5fcd6539d298e25967b983601226c1e20485c9dbc9201a88fa37f8f"}
Nov 03 21:29:14.879447 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.879245    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wk2lw" event={"ID":"af4170fc-093a-4066-bf79-4f57388f073d","Type":"ContainerStarted","Data":"fb1b078a90cb0432fc2929c8bf26878a7811f4ff102efd3c361e6deb32e3cbd0"}
Nov 03 21:29:14.883846 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.883780    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swgh5" event={"ID":"b7e2c320-fbf4-40e6-938d-b4e1377d769c","Type":"ContainerStarted","Data":"7cd88de0cca83cd280cedb54ec74f8fc1e88469fbb76f7999c8e8d74bc84bdce"}
Nov 03 21:29:14.889659 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.888479    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x55l2" event={"ID":"47bb3d82-52e3-4e39-8b1a-1deb886a29db","Type":"ContainerStarted","Data":"1754f54422e37a4e203375cd9ba0dc715ee426d9647fb668760b3e0299c3b742"}
Nov 03 21:29:14.903287 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.903014    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wjgz5" event={"ID":"3a287e20-e2d0-451a-97ce-b4fddffeb788","Type":"ContainerStarted","Data":"03f2395f5e0075fcc939aa1646b0cb4da91a99c2815ed1b182b4ce31dd7107d0"}
Nov 03 21:29:14.913136 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.913109    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-529hb" event={"ID":"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70","Type":"ContainerStarted","Data":"155e9bae8c395f1c283b407f9e61547fa8dfd5178e22a3ad8c814f28fa8061c5"}
Nov 03 21:29:14.918353 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:14.918324    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk" event={"ID":"aa5882bd-1e86-428d-afec-490ebd2da878","Type":"ContainerStarted","Data":"b3ea48873e2ce64601b2cb0af1686e3d7a0231bb0c3f3a61a6c5a05947a9fddf"}
Nov 03 21:29:15.314426 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:15.314379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:15.314618 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:15.314482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:15.314694 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:15.314623    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 21:29:15.314694 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:15.314690    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret podName:b978b73d-666a-4c25-b959-74d3e14c7bbf nodeName:}" failed. No retries permitted until 2026-11-03 21:29:17.314671941 +0000 UTC m=+6.124607377 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret") pod "global-pull-secret-syncer-vmr9k" (UID: "b978b73d-666a-4c25-b959-74d3e14c7bbf") : object "kube-system"/"original-pull-secret" not registered
Nov 03 21:29:15.315142 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:15.315076    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 21:29:15.315142 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:15.315129    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs podName:ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a nodeName:}" failed. No retries permitted until 2026-11-03 21:29:17.315113604 +0000 UTC m=+6.125049020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs") pod "network-metrics-daemon-4gsg9" (UID: "ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 21:29:15.415739 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:15.415650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fh5b\" (UniqueName: \"kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b\") pod \"network-check-target-7s62j\" (UID: \"29ad1278-e154-41fb-8648-0adc8d7a648f\") " pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:15.415881 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:15.415867    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 21:29:15.415955 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:15.415892    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 21:29:15.415955 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:15.415906    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2fh5b for pod openshift-network-diagnostics/network-check-target-7s62j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 21:29:15.416059 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:15.415966    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b podName:29ad1278-e154-41fb-8648-0adc8d7a648f nodeName:}" failed. No retries permitted until 2026-11-03 21:29:17.415947143 +0000 UTC m=+6.225882562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2fh5b" (UniqueName: "kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b") pod "network-check-target-7s62j" (UID: "29ad1278-e154-41fb-8648-0adc8d7a648f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 21:29:15.633097 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:15.633031    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 21:24:13 +0000 UTC" deadline="2028-04-29 23:24:56.34260064 +0000 UTC"
Nov 03 21:29:15.633097 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:15.633097    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13033h55m40.709508727s"
Nov 03 21:29:15.829709 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:15.829643    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:15.829861 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:15.829780    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:15.941310 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:15.940208    2580 generic.go:356] "Generic (PLEG): container finished" podID="86212ec82a64966b4f712c3fdb2b1cf2" containerID="1893520b5838a40c7096adede4496971154d47e96558f015dff62011774187b6" exitCode=0
Nov 03 21:29:15.941310 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:15.940529    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal" event={"ID":"86212ec82a64966b4f712c3fdb2b1cf2","Type":"ContainerDied","Data":"1893520b5838a40c7096adede4496971154d47e96558f015dff62011774187b6"}
Nov 03 21:29:15.951677 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:15.951642    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal" event={"ID":"9caeb2eb053e15a50e91a5dd1ebbc6a1","Type":"ContainerStarted","Data":"ff2bb96756e497427a60d202bf6c9f784834093f26742caa64ac9d7ee0c57d76"}
Nov 03 21:29:16.829899 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:16.829849    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:16.830331 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:16.829974    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:16.830394 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:16.830348    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:16.830458 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:16.830435    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:16.958590 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:16.958486    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal" event={"ID":"86212ec82a64966b4f712c3fdb2b1cf2","Type":"ContainerStarted","Data":"1274d702f109eea62c4f78eba164e52566ade1207a2f3f8a6793c540578884a9"}
Nov 03 21:29:16.963410 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:16.963369    2580 generic.go:356] "Generic (PLEG): container finished" podID="b7e2c320-fbf4-40e6-938d-b4e1377d769c" containerID="a38cac1102562df23d22b196259b451a7dac03d5b07e9c16cd5e503c6a719472" exitCode=0
Nov 03 21:29:16.964336 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:16.964308    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swgh5" event={"ID":"b7e2c320-fbf4-40e6-938d-b4e1377d769c","Type":"ContainerDied","Data":"a38cac1102562df23d22b196259b451a7dac03d5b07e9c16cd5e503c6a719472"}
Nov 03 21:29:16.998194 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:16.997938    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal" podStartSLOduration=3.9979189760000002 podStartE2EDuration="3.997918976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:29:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:16.02906335 +0000 UTC m=+4.838998786" watchObservedRunningTime="2026-11-03 21:29:16.997918976 +0000 UTC m=+5.807854412"
Nov 03 21:29:17.039182 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:17.039083    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal" podStartSLOduration=4.039063931 podStartE2EDuration="4.039063931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:29:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:16.99824638 +0000 UTC m=+5.808181816" watchObservedRunningTime="2026-11-03 21:29:17.039063931 +0000 UTC m=+5.848999368"
Nov 03 21:29:17.335726 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:17.335499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:17.335877 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:17.335743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:17.345694 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:17.345660    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 21:29:17.345864 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:17.345750    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret podName:b978b73d-666a-4c25-b959-74d3e14c7bbf nodeName:}" failed. No retries permitted until 2026-11-03 21:29:21.345726521 +0000 UTC m=+10.155661955 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret") pod "global-pull-secret-syncer-vmr9k" (UID: "b978b73d-666a-4c25-b959-74d3e14c7bbf") : object "kube-system"/"original-pull-secret" not registered
Nov 03 21:29:17.345975 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:17.345940    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 21:29:17.346040 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:17.346010    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs podName:ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a nodeName:}" failed. No retries permitted until 2026-11-03 21:29:21.345987281 +0000 UTC m=+10.155922694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs") pod "network-metrics-daemon-4gsg9" (UID: "ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 21:29:17.437124 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:17.437082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fh5b\" (UniqueName: \"kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b\") pod \"network-check-target-7s62j\" (UID: \"29ad1278-e154-41fb-8648-0adc8d7a648f\") " pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:17.437295 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:17.437273    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 21:29:17.437295 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:17.437290    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 21:29:17.437398 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:17.437302    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2fh5b for pod openshift-network-diagnostics/network-check-target-7s62j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 21:29:17.437398 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:17.437373    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b podName:29ad1278-e154-41fb-8648-0adc8d7a648f nodeName:}" failed. No retries permitted until 2026-11-03 21:29:21.437353331 +0000 UTC m=+10.247288751 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2fh5b" (UniqueName: "kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b") pod "network-check-target-7s62j" (UID: "29ad1278-e154-41fb-8648-0adc8d7a648f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 21:29:17.836116 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:17.836090    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:17.836403 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:17.836217    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:17.976610 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:17.976462    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-79tpv" event={"ID":"45263f32-066f-4bdf-84b9-a3ebd1f33f90","Type":"ContainerStarted","Data":"94bfe94fc6172cd2b8394dc546fe8a3a3702e4ded7a139cf2f22fcb03e06c6fb"}
Nov 03 21:29:18.828814 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:18.828784    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:18.828974 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:18.828918    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:18.829045 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:18.829005    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:18.829099 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:18.829083    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:18.982375 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:18.982323    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wjgz5" event={"ID":"3a287e20-e2d0-451a-97ce-b4fddffeb788","Type":"ContainerStarted","Data":"f42f2866c6e27561a45cdc81a5ec403c697b911ee044e1fe320042695d03d9a5"}
Nov 03 21:29:19.019456 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:19.019396    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-79tpv" podStartSLOduration=3.324619488 podStartE2EDuration="7.01937457s" totalImagesPullingTime="3.694755082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:18.013424532 +0000 UTC m=+6.823359970" watchObservedRunningTime="2026-11-03 21:29:19.01937457 +0000 UTC m=+7.829310007"
Nov 03 21:29:19.828863 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:19.828835    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:19.828995 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:19.828973    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:19.987942 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:19.987858    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk" event={"ID":"aa5882bd-1e86-428d-afec-490ebd2da878","Type":"ContainerStarted","Data":"3ab0616918093d4f9854e242800cc2e3a21fd21bb44cc2528ad9313c95c9144b"}
Nov 03 21:29:20.328098 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:20.328066    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:20.379244 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:20.379140    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:20.379760 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:20.379734    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:20.416362 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:20.416305    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wjgz5" podStartSLOduration=3.795727447 podStartE2EDuration="8.416284771s" totalImagesPullingTime="4.620557324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:19.019708344 +0000 UTC m=+7.829643781" watchObservedRunningTime="2026-11-03 21:29:20.416284771 +0000 UTC m=+9.226220211"
Nov 03 21:29:20.829372 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:20.829330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:20.829590 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:20.829489    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:20.829952 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:20.829929    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:20.830058 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:20.830038    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:20.992017 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:20.991988    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:21.263833 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:21.263199    2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 21:29:21.370873 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:21.370847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:21.370990 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:21.370920    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:21.371065 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:21.371005    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 21:29:21.371065 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:21.371053    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret podName:b978b73d-666a-4c25-b959-74d3e14c7bbf nodeName:}" failed. No retries permitted until 2026-11-03 21:29:29.371040318 +0000 UTC m=+18.180975745 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret") pod "global-pull-secret-syncer-vmr9k" (UID: "b978b73d-666a-4c25-b959-74d3e14c7bbf") : object "kube-system"/"original-pull-secret" not registered
Nov 03 21:29:21.371176 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:21.371005    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 21:29:21.371176 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:21.371121    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs podName:ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a nodeName:}" failed. No retries permitted until 2026-11-03 21:29:29.371107808 +0000 UTC m=+18.181043236 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs") pod "network-metrics-daemon-4gsg9" (UID: "ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 21:29:21.471707 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:21.471681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fh5b\" (UniqueName: \"kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b\") pod \"network-check-target-7s62j\" (UID: \"29ad1278-e154-41fb-8648-0adc8d7a648f\") " pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:21.471843 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:21.471828    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 21:29:21.471909 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:21.471849    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 21:29:21.471909 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:21.471859    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2fh5b for pod openshift-network-diagnostics/network-check-target-7s62j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 21:29:21.471909 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:21.471900    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b podName:29ad1278-e154-41fb-8648-0adc8d7a648f nodeName:}" failed. No retries permitted until 2026-11-03 21:29:29.47188761 +0000 UTC m=+18.281823023 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2fh5b" (UniqueName: "kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b") pod "network-check-target-7s62j" (UID: "29ad1278-e154-41fb-8648-0adc8d7a648f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 21:29:21.773708 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:21.773086    2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T21:29:21.263212117Z","UUID":"b811efb2-fcb5-48eb-b71e-dcf4536fac99","Handler":null,"Name":"","Endpoint":""}
Nov 03 21:29:21.777757 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:21.777677    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 21:29:21.777757 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:21.777709    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 21:29:21.830535 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:21.828582    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:21.830535 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:21.829708    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:21.995153 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:21.995120    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-529hb" event={"ID":"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70","Type":"ContainerStarted","Data":"8856340b130078cf8f03f8a4847d3fc209f4e78e28d18ce93c392edccec7c158"}
Nov 03 21:29:22.002786 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:22.002757    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk" event={"ID":"aa5882bd-1e86-428d-afec-490ebd2da878","Type":"ContainerStarted","Data":"5150d25a4be8a2c45435ce40e09a699848f09abc142418533f8f443d8ac67bdb"}
Nov 03 21:29:22.007208 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:22.007179    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q2zcj" event={"ID":"8bcb1ef0-aa94-4932-99a6-53178e770a68","Type":"ContainerStarted","Data":"0400fe955ab949dcbcb7a30e30b214018bdcce824831f37e41e5fc29bfee6f50"}
Nov 03 21:29:22.011053 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:22.011025    2580 generic.go:356] "Generic (PLEG): container finished" podID="b7e2c320-fbf4-40e6-938d-b4e1377d769c" containerID="301c335e7e75b5c49d42e611674255211f49cb22278bb08611aad53df345629e" exitCode=0
Nov 03 21:29:22.011974 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:22.011950    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swgh5" event={"ID":"b7e2c320-fbf4-40e6-938d-b4e1377d769c","Type":"ContainerDied","Data":"301c335e7e75b5c49d42e611674255211f49cb22278bb08611aad53df345629e"}
Nov 03 21:29:22.077215 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:22.077160    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-529hb" podStartSLOduration=2.297600612 podStartE2EDuration="10.077140594s" totalImagesPullingTime="7.779539982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:22.035203132 +0000 UTC m=+10.845138570" watchObservedRunningTime="2026-11-03 21:29:22.077140594 +0000 UTC m=+10.887076031"
Nov 03 21:29:22.115177 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:22.115117    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-q2zcj" podStartSLOduration=2.719771463 podStartE2EDuration="10.115099449s" totalImagesPullingTime="7.395327986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:22.11429335 +0000 UTC m=+10.924228790" watchObservedRunningTime="2026-11-03 21:29:22.115099449 +0000 UTC m=+10.925034886"
Nov 03 21:29:22.829266 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:22.828978    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:22.829409 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:22.829333    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:22.829409 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:22.829375    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:22.829524 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:22.829440    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:23.016069 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:23.016036    2580 generic.go:356] "Generic (PLEG): container finished" podID="b7e2c320-fbf4-40e6-938d-b4e1377d769c" containerID="7cb88288417ee31ee8816573830bc20ad8984152ee9950d1483030f7de66533a" exitCode=0
Nov 03 21:29:23.016534 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:23.016141    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swgh5" event={"ID":"b7e2c320-fbf4-40e6-938d-b4e1377d769c","Type":"ContainerDied","Data":"7cb88288417ee31ee8816573830bc20ad8984152ee9950d1483030f7de66533a"}
Nov 03 21:29:23.019620 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:23.019588    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk" event={"ID":"aa5882bd-1e86-428d-afec-490ebd2da878","Type":"ContainerStarted","Data":"3256248558f4ed23fce48c60d89d8371c481469a21c44d0ba1fc26d3d6275625"}
Nov 03 21:29:23.829155 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:23.829125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:23.829312 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:23.829246    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:24.023052 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:24.022974    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wk2lw" event={"ID":"af4170fc-093a-4066-bf79-4f57388f073d","Type":"ContainerStarted","Data":"2ed456e52b2dafa61a9f9b6541f51412e3ecd432ba2ab18980fcd617261ad5e8"}
Nov 03 21:29:24.027232 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:24.027199    2580 generic.go:356] "Generic (PLEG): container finished" podID="b7e2c320-fbf4-40e6-938d-b4e1377d769c" containerID="4711f9d2a4cb168aea7099fb34e153503b9ecea8a82e92b8be9ad341fa039c86" exitCode=0
Nov 03 21:29:24.027473 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:24.027446    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swgh5" event={"ID":"b7e2c320-fbf4-40e6-938d-b4e1377d769c","Type":"ContainerDied","Data":"4711f9d2a4cb168aea7099fb34e153503b9ecea8a82e92b8be9ad341fa039c86"}
Nov 03 21:29:24.031874 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:24.031854    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x55l2" event={"ID":"47bb3d82-52e3-4e39-8b1a-1deb886a29db","Type":"ContainerStarted","Data":"2a48aa06f2bbf8a66497fb9ecd8baadece073e749b70a1dbc629fed343b3cbc8"}
Nov 03 21:29:24.060058 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:24.060011    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wk2lw" podStartSLOduration=4.038569574 podStartE2EDuration="12.059996026s" totalImagesPullingTime="8.021426452s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:24.059597565 +0000 UTC m=+12.869532999" watchObservedRunningTime="2026-11-03 21:29:24.059996026 +0000 UTC m=+12.869931479"
Nov 03 21:29:24.060210 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:24.060136    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk" podStartSLOduration=4.139282435 podStartE2EDuration="12.060129919s" totalImagesPullingTime="7.920847484s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:29:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:23.096167383 +0000 UTC m=+11.906102819" watchObservedRunningTime="2026-11-03 21:29:24.060129919 +0000 UTC m=+12.870065359"
Nov 03 21:29:24.100787 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:24.100734    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-x55l2" podStartSLOduration=2.330436493 podStartE2EDuration="12.100721969s" totalImagesPullingTime="9.770285476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:24.100716859 +0000 UTC m=+12.910652297" watchObservedRunningTime="2026-11-03 21:29:24.100721969 +0000 UTC m=+12.910657400"
Nov 03 21:29:24.828639 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:24.828608    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:24.828824 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:24.828649    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:24.828824 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:24.828735    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:24.828915 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:24.828825    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:25.831977 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:25.831949    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:25.832626 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:25.832074    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:26.038314 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:26.038236    2580 generic.go:356] "Generic (PLEG): container finished" podID="b7e2c320-fbf4-40e6-938d-b4e1377d769c" containerID="4a4a93ccbc8e63a866961e4e83df497c408fc1867609942f0899607a01daab4c" exitCode=0
Nov 03 21:29:26.038314 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:26.038286    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swgh5" event={"ID":"b7e2c320-fbf4-40e6-938d-b4e1377d769c","Type":"ContainerDied","Data":"4a4a93ccbc8e63a866961e4e83df497c408fc1867609942f0899607a01daab4c"}
Nov 03 21:29:26.828703 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:26.828665    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:26.828703 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:26.828706    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:26.828924 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:26.828810    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:26.828979 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:26.828917    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:27.044170 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:27.044128    2580 generic.go:356] "Generic (PLEG): container finished" podID="b7e2c320-fbf4-40e6-938d-b4e1377d769c" containerID="3561b6f98e60f401ecb00997ac36571471777fe063a86eb21e2d01b199cf20ff" exitCode=0
Nov 03 21:29:27.044785 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:27.044200    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swgh5" event={"ID":"b7e2c320-fbf4-40e6-938d-b4e1377d769c","Type":"ContainerDied","Data":"3561b6f98e60f401ecb00997ac36571471777fe063a86eb21e2d01b199cf20ff"}
Nov 03 21:29:27.829246 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:27.829079    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:27.829422 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:27.829330    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:28.048885 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:28.048854    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swgh5" event={"ID":"b7e2c320-fbf4-40e6-938d-b4e1377d769c","Type":"ContainerStarted","Data":"f2fe49f4aff23cb99c18ac5df34650e42123a9caa305906f2acc5718e4e305e4"}
Nov 03 21:29:28.115676 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:28.115631    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-swgh5" podStartSLOduration=6.300335152 podStartE2EDuration="16.115616392s" totalImagesPullingTime="9.81528124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:29:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:28.102522338 +0000 UTC m=+16.912457764" watchObservedRunningTime="2026-11-03 21:29:28.115616392 +0000 UTC m=+16.925551827"
Nov 03 21:29:28.829250 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:28.829046    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:28.829250 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:28.829068    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:28.829441 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:28.829294    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:28.829441 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:28.829392    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:29.053493 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:29.053467    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:29:29.053880 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:29.053791    2580 generic.go:356] "Generic (PLEG): container finished" podID="fe26d35e-47a4-497a-9fe1-64cc807267e9" containerID="8e26501d16b99372702bdec937ea4fb8522f8c991892eb6b00ac346ad622ec33" exitCode=1
Nov 03 21:29:29.053880 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:29.053866    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq" event={"ID":"fe26d35e-47a4-497a-9fe1-64cc807267e9","Type":"ContainerStarted","Data":"71cec44757267b8020a7d4717098fe2fedfeaed870abbf5069aa0e711f36ebf8"}
Nov 03 21:29:29.053968 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:29.053931    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq" event={"ID":"fe26d35e-47a4-497a-9fe1-64cc807267e9","Type":"ContainerStarted","Data":"9cfcb0c7fed99546087b536776439800d83d344c489f63b2d881efbf4a833f01"}
Nov 03 21:29:29.054003 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:29.053977    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq" event={"ID":"fe26d35e-47a4-497a-9fe1-64cc807267e9","Type":"ContainerStarted","Data":"94e9184eeb5d8028c6b22da66a511e5dc25b3870e90ba7f28e89cde51f51b002"}
Nov 03 21:29:29.054032 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:29.054007    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq" event={"ID":"fe26d35e-47a4-497a-9fe1-64cc807267e9","Type":"ContainerStarted","Data":"ed62ff522bc2a19692c02f5d4a9d5f917eb961aeaec0b5bda5080de6638aa777"}
Nov 03 21:29:29.054060 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:29.054041    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq" event={"ID":"fe26d35e-47a4-497a-9fe1-64cc807267e9","Type":"ContainerDied","Data":"8e26501d16b99372702bdec937ea4fb8522f8c991892eb6b00ac346ad622ec33"}
Nov 03 21:29:29.054090 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:29.054071    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq" event={"ID":"fe26d35e-47a4-497a-9fe1-64cc807267e9","Type":"ContainerStarted","Data":"81197b09154e690eed1efc8423119ae3696a3ffd4ad90e585bbe6a41071293dc"}
Nov 03 21:29:29.431580 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:29.431546    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:29.431742 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:29.431609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:29.431742 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:29.431692    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 21:29:29.431742 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:29.431717    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 21:29:29.431742 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:29.431743    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret podName:b978b73d-666a-4c25-b959-74d3e14c7bbf nodeName:}" failed. No retries permitted until 2026-11-03 21:29:45.431730607 +0000 UTC m=+34.241666021 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret") pod "global-pull-secret-syncer-vmr9k" (UID: "b978b73d-666a-4c25-b959-74d3e14c7bbf") : object "kube-system"/"original-pull-secret" not registered
Nov 03 21:29:29.431876 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:29.431775    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs podName:ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a nodeName:}" failed. No retries permitted until 2026-11-03 21:29:45.4317624 +0000 UTC m=+34.241697818 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs") pod "network-metrics-daemon-4gsg9" (UID: "ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 21:29:29.532348 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:29.532315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fh5b\" (UniqueName: \"kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b\") pod \"network-check-target-7s62j\" (UID: \"29ad1278-e154-41fb-8648-0adc8d7a648f\") " pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:29.532492 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:29.532455    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 21:29:29.532492 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:29.532473    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 21:29:29.532492 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:29.532484    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2fh5b for pod openshift-network-diagnostics/network-check-target-7s62j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 21:29:29.532696 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:29.532560    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b podName:29ad1278-e154-41fb-8648-0adc8d7a648f nodeName:}" failed. No retries permitted until 2026-11-03 21:29:45.532542086 +0000 UTC m=+34.342477514 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2fh5b" (UniqueName: "kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b") pod "network-check-target-7s62j" (UID: "29ad1278-e154-41fb-8648-0adc8d7a648f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 21:29:29.828845 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:29.828769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:29.828973 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:29.828868    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:30.829169 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:30.829137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:30.829169 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:30.829169    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:30.829684 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:30.829246    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:30.829684 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:30.829323    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:31.061282 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:31.061213    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:29:31.061574 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:31.061550    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq" event={"ID":"fe26d35e-47a4-497a-9fe1-64cc807267e9","Type":"ContainerStarted","Data":"f6cea347c8073f70eebaf39c8e492672fb3e626da64cf2220643fa40a6bed374"}
Nov 03 21:29:31.829376 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:31.829346    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:31.829759 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:31.829432    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:32.828497 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:32.828459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:32.828678 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:32.828580    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:32.828678 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:32.828630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:32.828789 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:32.828718    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:33.068227 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:33.068157    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:29:33.068842 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:33.068522    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq" event={"ID":"fe26d35e-47a4-497a-9fe1-64cc807267e9","Type":"ContainerStarted","Data":"b84949804ac381ac55afefca9d9c823701e3860b475c73b5e17eb695e6630691"}
Nov 03 21:29:33.068842 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:33.068734    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:33.068960 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:33.068879    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:33.068960 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:33.068900    2580 scope.go:122] "RemoveContainer" containerID="8e26501d16b99372702bdec937ea4fb8522f8c991892eb6b00ac346ad622ec33"
Nov 03 21:29:33.068960 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:33.068950    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:33.084147 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:33.084128    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:33.084553 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:33.084536    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:33.543701 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:33.543672    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:33.692443 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:33.692414    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:33.828927 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:33.828863    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:33.829048 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:33.828970    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:34.074217 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:34.074190    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:29:34.074592 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:34.074564    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq" event={"ID":"fe26d35e-47a4-497a-9fe1-64cc807267e9","Type":"ContainerStarted","Data":"e01031a62e080e9f4aae1d337d914df992228c104457c398495abcbb9aa41c77"}
Nov 03 21:29:34.150285 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:34.150252    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:34.162267 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:34.162231    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq" podStartSLOduration=7.988473891 podStartE2EDuration="22.162218073s" totalImagesPullingTime="14.173744182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:34.162014435 +0000 UTC m=+22.971949871" watchObservedRunningTime="2026-11-03 21:29:34.162218073 +0000 UTC m=+22.972153550"
Nov 03 21:29:34.248300 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:34.248257    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:34.824634 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:34.824400    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vmr9k"]
Nov 03 21:29:34.824802 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:34.824739    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:34.824878 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:34.824837    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:34.827494 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:34.827471    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7s62j"]
Nov 03 21:29:34.827600 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:34.827577    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:34.827669 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:34.827653    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:34.827936 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:34.827920    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4gsg9"]
Nov 03 21:29:34.828008 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:34.827999    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:34.828082 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:34.828069    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:36.828675 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:36.828646    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:36.828675 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:36.828675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:36.829070 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:36.828686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:36.829070 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:36.828756    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:36.829070 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:36.828866    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:36.829070 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:36.828925    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:37.019265 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:37.019240    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:37.116681 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:37.116648    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:38.829329 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:38.829298    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:38.829763 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:38.829337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:38.829763 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:38.829312    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:38.829763 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:38.829401    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:38.829763 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:38.829467    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:38.829763 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:38.829535    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:39.789044 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:39.789015    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:39.886478 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:39.886442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:39.984624 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:39.984597    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:40.081482 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:40.081401    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:40.543852 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:40.543806    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:40.671936 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:40.671898    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:40.771859 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:40.771829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:40.829107 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:40.829038    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:40.829107 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:40.829058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:40.829259 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:40.829147    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:40.829259 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:40.829168    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gsg9" podUID="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a"
Nov 03 21:29:40.829259 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:40.829214    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vmr9k" podUID="b978b73d-666a-4c25-b959-74d3e14c7bbf"
Nov 03 21:29:40.829354 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:40.829286    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7s62j" podUID="29ad1278-e154-41fb-8648-0adc8d7a648f"
Nov 03 21:29:40.872592 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:40.872566    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:40.947342 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:40.947320    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeReady"
Nov 03 21:29:40.947689 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:40.947446    2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 03 21:29:40.999362 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:40.999338    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kbh4v"]
Nov 03 21:29:41.008647 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.008631    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kbh4v"
Nov 03 21:29:41.018704 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.018684    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-prbdm"]
Nov 03 21:29:41.053741 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.053717    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kbh4v"]
Nov 03 21:29:41.053741 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.053750    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sz7tz"]
Nov 03 21:29:41.053906 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.053831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.058913 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.058888    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-prbdm"]
Nov 03 21:29:41.059128 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.059105    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.061809 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.061788    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sz7tz"]
Nov 03 21:29:41.070177 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.070159    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 21:29:41.070277 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.070265    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 21:29:41.070417 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.070403    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 21:29:41.070487 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.070445    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 21:29:41.070487 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.070481    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 21:29:41.073442 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.073427    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 21:29:41.073667 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.073647    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 21:29:41.073748 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.073686    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 21:29:41.073748 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.073709    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-fv5g9\""
Nov 03 21:29:41.073748 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.073725    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qcvh8\""
Nov 03 21:29:41.073748 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.073727    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 21:29:41.074750 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.074738    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qc6g7\""
Nov 03 21:29:41.124741 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.124717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15567c09-c50f-4c3b-81b5-5357f08fa543-cert\") pod \"ingress-canary-kbh4v\" (UID: \"15567c09-c50f-4c3b-81b5-5357f08fa543\") " pod="openshift-ingress-canary/ingress-canary-kbh4v"
Nov 03 21:29:41.124859 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.124755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tctvw\" (UniqueName: \"kubernetes.io/projected/15567c09-c50f-4c3b-81b5-5357f08fa543-kube-api-access-tctvw\") pod \"ingress-canary-kbh4v\" (UID: \"15567c09-c50f-4c3b-81b5-5357f08fa543\") " pod="openshift-ingress-canary/ingress-canary-kbh4v"
Nov 03 21:29:41.225525 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.225485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.225667 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.225547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-data-volume\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.225667 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.225567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qcc8\" (UniqueName: \"kubernetes.io/projected/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-kube-api-access-6qcc8\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.225667 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.225636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-tmp-dir\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.225763 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.225706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-config-volume\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.225763 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.225737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15567c09-c50f-4c3b-81b5-5357f08fa543-cert\") pod \"ingress-canary-kbh4v\" (UID: \"15567c09-c50f-4c3b-81b5-5357f08fa543\") " pod="openshift-ingress-canary/ingress-canary-kbh4v"
Nov 03 21:29:41.225830 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.225820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tctvw\" (UniqueName: \"kubernetes.io/projected/15567c09-c50f-4c3b-81b5-5357f08fa543-kube-api-access-tctvw\") pod \"ingress-canary-kbh4v\" (UID: \"15567c09-c50f-4c3b-81b5-5357f08fa543\") " pod="openshift-ingress-canary/ingress-canary-kbh4v"
Nov 03 21:29:41.225879 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.225844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-crio-socket\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.225879 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.225863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.226036 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.225895    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.226036 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.225984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgc4m\" (UniqueName: \"kubernetes.io/projected/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-api-access-jgc4m\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.229208 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.229191    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15567c09-c50f-4c3b-81b5-5357f08fa543-cert\") pod \"ingress-canary-kbh4v\" (UID: \"15567c09-c50f-4c3b-81b5-5357f08fa543\") " pod="openshift-ingress-canary/ingress-canary-kbh4v"
Nov 03 21:29:41.245639 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.245613    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tctvw\" (UniqueName: \"kubernetes.io/projected/15567c09-c50f-4c3b-81b5-5357f08fa543-kube-api-access-tctvw\") pod \"ingress-canary-kbh4v\" (UID: \"15567c09-c50f-4c3b-81b5-5357f08fa543\") " pod="openshift-ingress-canary/ingress-canary-kbh4v"
Nov 03 21:29:41.326534 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.326491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-data-volume\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.326666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.326541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qcc8\" (UniqueName: \"kubernetes.io/projected/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-kube-api-access-6qcc8\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.326666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.326559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-tmp-dir\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.326666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.326587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-config-volume\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.326666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.326625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-crio-socket\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.326666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.326641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.326666 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.326661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.326945 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:41.326720    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 03 21:29:41.326945 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:41.326781    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm podName:ff9199eb-c6a1-4a45-9b7b-862b6e778f4b nodeName:}" failed. No retries permitted until 2026-11-03 21:29:41.826764797 +0000 UTC m=+30.636700227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-prbdm" (UID: "ff9199eb-c6a1-4a45-9b7b-862b6e778f4b") : configmap "kube-rbac-proxy" not found
Nov 03 21:29:41.326945 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.326772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgc4m\" (UniqueName: \"kubernetes.io/projected/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-api-access-jgc4m\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.326945 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:41.326788    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 21:29:41.326945 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.326824    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-crio-socket\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.326945 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.326827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.326945 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:41.326861    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls podName:ff9199eb-c6a1-4a45-9b7b-862b6e778f4b nodeName:}" failed. No retries permitted until 2026-11-03 21:29:41.826842112 +0000 UTC m=+30.636777533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-prbdm" (UID: "ff9199eb-c6a1-4a45-9b7b-862b6e778f4b") : secret "insights-runtime-extractor-tls" not found
Nov 03 21:29:41.326945 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:41.326884    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 21:29:41.326945 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.326891    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-data-volume\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.326945 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:41.326915    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls podName:d09b08fd-64c6-4b9e-ba06-273cbedbd58a nodeName:}" failed. No retries permitted until 2026-11-03 21:29:41.826905212 +0000 UTC m=+30.636840641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls") pod "dns-default-sz7tz" (UID: "d09b08fd-64c6-4b9e-ba06-273cbedbd58a") : secret "dns-default-metrics-tls" not found
Nov 03 21:29:41.327389 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.327054    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-tmp-dir\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.327389 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.327290    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-config-volume\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.344050 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.344028    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kbh4v"
Nov 03 21:29:41.346927 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.346903    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qcc8\" (UniqueName: \"kubernetes.io/projected/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-kube-api-access-6qcc8\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.347080 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.347058    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgc4m\" (UniqueName: \"kubernetes.io/projected/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-api-access-jgc4m\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.366555 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.366530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:41.690157 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.690125    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kbh4v"]
Nov 03 21:29:41.693849 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:41.693825    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15567c09_c50f_4c3b_81b5_5357f08fa543.slice/crio-47fd279add685d3efc6605df15dd287721c3f8b3b7ac04c68a1010dff9858590 WatchSource:0}: Error finding container 47fd279add685d3efc6605df15dd287721c3f8b3b7ac04c68a1010dff9858590: Status 404 returned error can't find the container with id 47fd279add685d3efc6605df15dd287721c3f8b3b7ac04c68a1010dff9858590
Nov 03 21:29:41.698949 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.698924    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:41.830923 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.830896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:41.831060 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.830995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.831060 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:41.831032    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 21:29:41.831155 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:41.831034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:41.831155 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:41.831087    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 03 21:29:41.831155 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:41.831114    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 21:29:41.831155 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:41.831091    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls podName:d09b08fd-64c6-4b9e-ba06-273cbedbd58a nodeName:}" failed. No retries permitted until 2026-11-03 21:29:42.831071551 +0000 UTC m=+31.641006979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls") pod "dns-default-sz7tz" (UID: "d09b08fd-64c6-4b9e-ba06-273cbedbd58a") : secret "dns-default-metrics-tls" not found
Nov 03 21:29:41.831291 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:41.831179    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm podName:ff9199eb-c6a1-4a45-9b7b-862b6e778f4b nodeName:}" failed. No retries permitted until 2026-11-03 21:29:42.831162857 +0000 UTC m=+31.641098284 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-prbdm" (UID: "ff9199eb-c6a1-4a45-9b7b-862b6e778f4b") : configmap "kube-rbac-proxy" not found
Nov 03 21:29:41.831291 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:41.831195    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls podName:ff9199eb-c6a1-4a45-9b7b-862b6e778f4b nodeName:}" failed. No retries permitted until 2026-11-03 21:29:42.831186687 +0000 UTC m=+31.641122106 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-prbdm" (UID: "ff9199eb-c6a1-4a45-9b7b-862b6e778f4b") : secret "insights-runtime-extractor-tls" not found
Nov 03 21:29:42.092783 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.092710    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kbh4v" event={"ID":"15567c09-c50f-4c3b-81b5-5357f08fa543","Type":"ContainerStarted","Data":"47fd279add685d3efc6605df15dd287721c3f8b3b7ac04c68a1010dff9858590"}
Nov 03 21:29:42.783200 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.783161    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:42.828653 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.828622    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:42.828818 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.828664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:42.828818 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.828654    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:42.840540 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.840499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:42.840669 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.840560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:42.840669 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.840605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:42.840669 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:42.840610    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 21:29:42.840669 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:42.840652    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 03 21:29:42.840669 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:42.840669    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls podName:ff9199eb-c6a1-4a45-9b7b-862b6e778f4b nodeName:}" failed. No retries permitted until 2026-11-03 21:29:44.840650149 +0000 UTC m=+33.650585575 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-prbdm" (UID: "ff9199eb-c6a1-4a45-9b7b-862b6e778f4b") : secret "insights-runtime-extractor-tls" not found
Nov 03 21:29:42.840880 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:42.840707    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm podName:ff9199eb-c6a1-4a45-9b7b-862b6e778f4b nodeName:}" failed. No retries permitted until 2026-11-03 21:29:44.840692097 +0000 UTC m=+33.650627511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-prbdm" (UID: "ff9199eb-c6a1-4a45-9b7b-862b6e778f4b") : configmap "kube-rbac-proxy" not found
Nov 03 21:29:42.840880 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:42.840717    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 21:29:42.840880 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:42.840758    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls podName:d09b08fd-64c6-4b9e-ba06-273cbedbd58a nodeName:}" failed. No retries permitted until 2026-11-03 21:29:44.840746882 +0000 UTC m=+33.650682310 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls") pod "dns-default-sz7tz" (UID: "d09b08fd-64c6-4b9e-ba06-273cbedbd58a") : secret "dns-default-metrics-tls" not found
Nov 03 21:29:42.845281 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.845258    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 21:29:42.845670 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.845651    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 21:29:42.845828 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.845810    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 21:29:42.845893 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.845841    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-rf9jq\""
Nov 03 21:29:42.845977 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.845955    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 21:29:42.846066 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.846051    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bxcrl\""
Nov 03 21:29:42.846125 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:42.846054    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-2kjq7\""
Nov 03 21:29:43.007944 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:43.007879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:43.106146 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:43.106109    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:43.205055 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:43.205018    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:43.377578 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:43.377491    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:44.237131 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:44.237106    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-529hb_0fe02b31-bc0e-4ffc-916d-f6cd60b68a70/dns-node-resolver/0.log"
Nov 03 21:29:44.422300 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:44.422272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:44.576580 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:44.576547    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:44.773849 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:44.773821    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:44.857563 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:44.857538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:44.857717 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:44.857603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:44.857717 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:44.857626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:44.857717 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:44.857678    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 21:29:44.857867 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:44.857737    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls podName:d09b08fd-64c6-4b9e-ba06-273cbedbd58a nodeName:}" failed. No retries permitted until 2026-11-03 21:29:48.857722649 +0000 UTC m=+37.667658063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls") pod "dns-default-sz7tz" (UID: "d09b08fd-64c6-4b9e-ba06-273cbedbd58a") : secret "dns-default-metrics-tls" not found
Nov 03 21:29:44.857867 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:44.857756    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 21:29:44.857867 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:44.857817    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls podName:ff9199eb-c6a1-4a45-9b7b-862b6e778f4b nodeName:}" failed. No retries permitted until 2026-11-03 21:29:48.857799027 +0000 UTC m=+37.667734457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-prbdm" (UID: "ff9199eb-c6a1-4a45-9b7b-862b6e778f4b") : secret "insights-runtime-extractor-tls" not found
Nov 03 21:29:44.858124 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:44.858098    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:44.914516 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:44.914488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:45.013562 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.013531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:45.103288 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.103256    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kbh4v" event={"ID":"15567c09-c50f-4c3b-81b5-5357f08fa543","Type":"ContainerStarted","Data":"0be552f7eaf664635011b105310165a680faa56a58d438232d7809826185a437"}
Nov 03 21:29:45.122677 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.122609    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:45.141653 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.141612    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kbh4v" podStartSLOduration=2.514420584 podStartE2EDuration="5.141600163s" totalImagesPullingTime="2.627179579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:45.140950346 +0000 UTC m=+33.950885777" watchObservedRunningTime="2026-11-03 21:29:45.141600163 +0000 UTC m=+33.951535626"
Nov 03 21:29:45.224075 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.224046    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:45.372616 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.372588    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:45.462011 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.461939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:45.462137 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.462013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:45.464215 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.464181    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:45.464296 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.464277    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:45.473149 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.473129    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:45.563333 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.563304    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fh5b\" (UniqueName: \"kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b\") pod \"network-check-target-7s62j\" (UID: \"29ad1278-e154-41fb-8648-0adc8d7a648f\") " pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:45.565568 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.565550    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fh5b\" (UniqueName: \"kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b\") pod \"network-check-target-7s62j\" (UID: \"29ad1278-e154-41fb-8648-0adc8d7a648f\") " pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:45.577589 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.577570    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:45.616880 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.616857    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:45.630743 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.630723    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:45.636141 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.636122    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wjgz5_3a287e20-e2d0-451a-97ce-b4fddffeb788/node-ca/0.log"
Nov 03 21:29:45.903417 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.903391    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7s62j"]
Nov 03 21:29:45.906234 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:45.906210    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ad1278_e154_41fb_8648_0adc8d7a648f.slice/crio-e09369fd42372eda0ccb8575fde6776bdcac58f8c9953cff6104442774ef77c4 WatchSource:0}: Error finding container e09369fd42372eda0ccb8575fde6776bdcac58f8c9953cff6104442774ef77c4: Status 404 returned error can't find the container with id e09369fd42372eda0ccb8575fde6776bdcac58f8c9953cff6104442774ef77c4
Nov 03 21:29:45.930313 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.930292    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4gsg9"]
Nov 03 21:29:45.933014 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:45.932992    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce0d1d4d_7bb3_42c7_ab87_6c5b460ce65a.slice/crio-82699a23ce8e807ce50d3091f372547c6efdc8a8b97b83844034c16c23e67409 WatchSource:0}: Error finding container 82699a23ce8e807ce50d3091f372547c6efdc8a8b97b83844034c16c23e67409: Status 404 returned error can't find the container with id 82699a23ce8e807ce50d3091f372547c6efdc8a8b97b83844034c16c23e67409
Nov 03 21:29:45.964748 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:45.964715    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vmr9k"]
Nov 03 21:29:45.967999 ip-10-0-137-212 kubenswrapper[2580]: W1103 21:29:45.967975    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb978b73d_666a_4c25_b959_74d3e14c7bbf.slice/crio-73cdd256c5d66f3cd132901d0cc1e43b746c929efed37dc7ff4b9abedd10a24e WatchSource:0}: Error finding container 73cdd256c5d66f3cd132901d0cc1e43b746c929efed37dc7ff4b9abedd10a24e: Status 404 returned error can't find the container with id 73cdd256c5d66f3cd132901d0cc1e43b746c929efed37dc7ff4b9abedd10a24e
Nov 03 21:29:46.037014 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:46.036954    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kbh4v_15567c09-c50f-4c3b-81b5-5357f08fa543/serve-healthcheck-canary/0.log"
Nov 03 21:29:46.106002 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:46.105963    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vmr9k" event={"ID":"b978b73d-666a-4c25-b959-74d3e14c7bbf","Type":"ContainerStarted","Data":"73cdd256c5d66f3cd132901d0cc1e43b746c929efed37dc7ff4b9abedd10a24e"}
Nov 03 21:29:46.107033 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:46.107002    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7s62j" event={"ID":"29ad1278-e154-41fb-8648-0adc8d7a648f","Type":"ContainerStarted","Data":"e09369fd42372eda0ccb8575fde6776bdcac58f8c9953cff6104442774ef77c4"}
Nov 03 21:29:46.108041 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:46.108024    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4gsg9" event={"ID":"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a","Type":"ContainerStarted","Data":"82699a23ce8e807ce50d3091f372547c6efdc8a8b97b83844034c16c23e67409"}
Nov 03 21:29:46.576629 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:46.576549    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:46.677430 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:46.677399    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:46.778226 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:46.778160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:46.984787 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:46.984754    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:47.174775 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:47.174701    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:47.375090 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:47.375055    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:47.703851 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:47.703826    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:48.116621 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:48.116484    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4gsg9" event={"ID":"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a","Type":"ContainerStarted","Data":"ac59d8bee596c68643819fa42b545a4e3814a56e11108edf0a85b97d85b8e081"}
Nov 03 21:29:48.116786 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:48.116687    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4gsg9" event={"ID":"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a","Type":"ContainerStarted","Data":"dc4e2b564052231e4e91f2fd09cb54bbdf56a3aabe3ed41ee19643f84eca0f01"}
Nov 03 21:29:48.156830 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:48.156612    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4gsg9" podStartSLOduration=34.487238124 podStartE2EDuration="36.156593324s" totalImagesPullingTime="1.6693552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:48.155346154 +0000 UTC m=+36.965281589" watchObservedRunningTime="2026-11-03 21:29:48.156593324 +0000 UTC m=+36.966528760"
Nov 03 21:29:48.574085 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:48.574003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:48.775868 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:48.775822    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:48.891920 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:48.891880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:48.892104 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:48.891968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:48.892104 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:48.892058    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 21:29:48.892201 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:48.892143    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls podName:d09b08fd-64c6-4b9e-ba06-273cbedbd58a nodeName:}" failed. No retries permitted until 2026-11-03 21:29:56.892122013 +0000 UTC m=+45.702057439 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls") pod "dns-default-sz7tz" (UID: "d09b08fd-64c6-4b9e-ba06-273cbedbd58a") : secret "dns-default-metrics-tls" not found
Nov 03 21:29:48.892201 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:48.892177    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 21:29:48.892275 ip-10-0-137-212 kubenswrapper[2580]: E1103 21:29:48.892231    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls podName:ff9199eb-c6a1-4a45-9b7b-862b6e778f4b nodeName:}" failed. No retries permitted until 2026-11-03 21:29:56.892214498 +0000 UTC m=+45.702149918 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-prbdm" (UID: "ff9199eb-c6a1-4a45-9b7b-862b6e778f4b") : secret "insights-runtime-extractor-tls" not found
Nov 03 21:29:48.974811 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:48.974777    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:49.146137 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:49.146043    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:49.247983 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:49.247949    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:49.347256 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:49.347231    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:49.449706 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:49.449680    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:50.123016 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:50.122976    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7s62j" event={"ID":"29ad1278-e154-41fb-8648-0adc8d7a648f","Type":"ContainerStarted","Data":"f1c89296eeeaad9af99a05b952be38f192212a5ee9b1281a0617d79b2a03daf9"}
Nov 03 21:29:50.123574 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:50.123118    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:50.161074 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:50.161016    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7s62j" podStartSLOduration=34.723931348 podStartE2EDuration="38.160997039s" totalImagesPullingTime="3.437065691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:50.159881863 +0000 UTC m=+38.969817298" watchObservedRunningTime="2026-11-03 21:29:50.160997039 +0000 UTC m=+38.970932479"
Nov 03 21:29:50.570615 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:50.570540    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:50.774468 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:50.774445    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:50.972053 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:50.972015    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:51.049014 ip-10-0-137-212 kubenswrapper[2580]: I1103 21:29:51.048893    2580 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 21:29:51.049057 ip-10-0-137-212 systemd[1]: Stopping Kubernetes Kubelet...
Nov 03 21:29:51.069863 ip-10-0-137-212 systemd[1]: kubelet.service: Deactivated successfully.
Nov 03 21:29:51.070081 ip-10-0-137-212 systemd[1]: Stopped Kubernetes Kubelet.
Nov 03 21:29:51.070140 ip-10-0-137-212 systemd[1]: kubelet.service: Consumed 2.327s CPU time.
Nov 03 21:29:51.083189 ip-10-0-137-212 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 21:29:51.150442 ip-10-0-137-212 kubenswrapper[4454]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 21:29:51.150442 ip-10-0-137-212 kubenswrapper[4454]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 21:29:51.150442 ip-10-0-137-212 kubenswrapper[4454]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 21:29:51.150442 ip-10-0-137-212 kubenswrapper[4454]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 21:29:51.152599 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152486    4454 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:51.152599 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152599    4454 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152603    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152606    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152610    4454 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152612    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152615    4454 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152618    4454 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152621    4454 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152624    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152626    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152629    4454 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152631    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152634    4454 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152637    4454 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152640    4454 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152642    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152645    4454 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152648    4454 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152651    4454 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152661    4454 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:51.152670 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152664    4454 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152667    4454 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152670    4454 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152672    4454 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152675    4454 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152678    4454 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152681    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152683    4454 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152686    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152689    4454 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152691    4454 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152694    4454 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152697    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152699    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152702    4454 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152704    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152707    4454 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152710    4454 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152713    4454 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152716    4454 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:51.153158 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152719    4454 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152722    4454 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152724    4454 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152727    4454 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152729    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152732    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152736    4454 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152739    4454 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152742    4454 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152745    4454 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152749    4454 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152754    4454 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152758    4454 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152762    4454 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152765    4454 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152768    4454 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152771    4454 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152774    4454 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152777    4454 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:51.153660 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152779    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152783    4454 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152785    4454 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152788    4454 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152790    4454 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152793    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152796    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152798    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152801    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152803    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152806    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152808    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152811    4454 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152814    4454 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152817    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152819    4454 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152822    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152825    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152827    4454 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152830    4454 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:51.154164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152833    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152835    4454 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152838    4454 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152841    4454 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152843    4454 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152846    4454 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152848    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152851    4454 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152853    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152856    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152859    4454 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152861    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152864    4454 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152867    4454 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152870    4454 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152872    4454 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152875    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152877    4454 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152880    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152882    4454 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:51.154667 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152885    4454 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152888    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152892    4454 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152896    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152899    4454 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.152902    4454 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153630    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153638    4454 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153641    4454 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153644    4454 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153646    4454 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153650    4454 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153653    4454 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153656    4454 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153659    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153661    4454 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153664    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153667    4454 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153669    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:51.155156 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153672    4454 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153675    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153677    4454 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153680    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153683    4454 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153686    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153688    4454 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153691    4454 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153693    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153696    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153699    4454 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153701    4454 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153703    4454 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153706    4454 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153708    4454 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153711    4454 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153714    4454 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153716    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153719    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153722    4454 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:51.155643 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153725    4454 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153727    4454 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153730    4454 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153733    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153736    4454 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153739    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153741    4454 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153746    4454 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153749    4454 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153752    4454 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153754    4454 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153757    4454 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153759    4454 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153762    4454 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153764    4454 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153767    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153769    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153772    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153775    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:51.156119 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153777    4454 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153780    4454 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153783    4454 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153785    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153788    4454 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153791    4454 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153793    4454 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153796    4454 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153799    4454 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153801    4454 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153804    4454 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153806    4454 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153809    4454 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153812    4454 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153814    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153817    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153820    4454 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153822    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153825    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153827    4454 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:51.156595 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153830    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153833    4454 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153837    4454 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153841    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153844    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153846    4454 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153849    4454 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153851    4454 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153854    4454 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153857    4454 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153859    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153862    4454 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153865    4454 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153867    4454 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153870    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153872    4454 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153875    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153877    4454 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153880    4454 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:51.157079 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153883    4454 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153885    4454 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153888    4454 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153891    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153894    4454 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153896    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153899    4454 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153902    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153905    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153908    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153910    4454 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153913    4454 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153915    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153917    4454 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.153920    4454 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154371    4454 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154376    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154379    4454 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154382    4454 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154385    4454 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:51.157543 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154387    4454 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154390    4454 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154393    4454 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154396    4454 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154398    4454 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154401    4454 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154403    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154406    4454 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154409    4454 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154411    4454 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154414    4454 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154416    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154419    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154422    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154425    4454 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154427    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154430    4454 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154434    4454 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154438    4454 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:51.158033 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154442    4454 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154445    4454 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154447    4454 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154450    4454 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154453    4454 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154455    4454 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154459    4454 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154462    4454 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154466    4454 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154468    4454 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154471    4454 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154474    4454 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154476    4454 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154479    4454 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154481    4454 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154484    4454 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154486    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154489    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154491    4454 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:51.158523 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154494    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154496    4454 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154499    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154501    4454 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154516    4454 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154519    4454 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154522    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154524    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154530    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154532    4454 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154535    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154538    4454 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154540    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154543    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154546    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154548    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154551    4454 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154553    4454 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154556    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154558    4454 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:51.159005 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154563    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154565    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154568    4454 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154570    4454 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154573    4454 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154576    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154578    4454 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154581    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154584    4454 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154587    4454 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154589    4454 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154591    4454 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154594    4454 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154597    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154599    4454 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154602    4454 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154604    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154606    4454 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154609    4454 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154611    4454 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:51.159516 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154615    4454 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154618    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154620    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154624    4454 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154627    4454 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154629    4454 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154632    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154635    4454 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154637    4454 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154640    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154642    4454 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154644    4454 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154648    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154651    4454 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154653    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154656    4454 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154658    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154662    4454 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154665    4454 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:51.160270 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154667    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154670    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154673    4454 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.154675    4454 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154771    4454 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154779    4454 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154784    4454 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154789    4454 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154794    4454 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154797    4454 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154801    4454 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154805    4454 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154808    4454 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154812    4454 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154816    4454 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154819    4454 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154822    4454 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154825    4454 flags.go:64] FLAG: --cgroup-root=""
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154828    4454 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154831    4454 flags.go:64] FLAG: --client-ca-file=""
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154834    4454 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154838    4454 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154842    4454 flags.go:64] FLAG: --cluster-domain=""
Nov 03 21:29:51.161030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154845    4454 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154848    4454 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154853    4454 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154857    4454 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154861    4454 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154864    4454 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154867    4454 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154870    4454 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154873    4454 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154877    4454 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154880    4454 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154883    4454 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154886    4454 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154890    4454 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154893    4454 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154896    4454 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154899    4454 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154902    4454 flags.go:64] FLAG: --enable-server="true"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154906    4454 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154910    4454 flags.go:64] FLAG: --event-burst="100"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154913    4454 flags.go:64] FLAG: --event-qps="50"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154916    4454 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154920    4454 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154923    4454 flags.go:64] FLAG: --eviction-hard=""
Nov 03 21:29:51.161619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154927    4454 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154930    4454 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154933    4454 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154936    4454 flags.go:64] FLAG: --eviction-soft=""
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154939    4454 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154942    4454 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154945    4454 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154948    4454 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154951    4454 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154954    4454 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154959    4454 flags.go:64] FLAG: --feature-gates=""
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154965    4454 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154969    4454 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154972    4454 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154975    4454 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154978    4454 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154981    4454 flags.go:64] FLAG: --help="false"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154984    4454 flags.go:64] FLAG: --hostname-override="ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154987    4454 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154990    4454 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154993    4454 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154996    4454 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.154999    4454 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155003    4454 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 21:29:51.162318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155006    4454 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155008    4454 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155011    4454 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155014    4454 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155018    4454 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155021    4454 flags.go:64] FLAG: --kube-reserved=""
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155024    4454 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155028    4454 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155031    4454 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155034    4454 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155036    4454 flags.go:64] FLAG: --lock-file=""
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155039    4454 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155042    4454 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155045    4454 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155051    4454 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155054    4454 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155058    4454 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155061    4454 flags.go:64] FLAG: --logging-format="text"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155066    4454 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155069    4454 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155072    4454 flags.go:64] FLAG: --manifest-url=""
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155075    4454 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155079    4454 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155082    4454 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155086    4454 flags.go:64] FLAG: --max-pods="110"
Nov 03 21:29:51.162912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155089    4454 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155093    4454 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155096    4454 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155098    4454 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155101    4454 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155104    4454 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155107    4454 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155114    4454 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155117    4454 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155120    4454 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155123    4454 flags.go:64] FLAG: --pod-cidr=""
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155126    4454 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155128    4454 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155132    4454 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155135    4454 flags.go:64] FLAG: --port="10250"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155138    4454 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155140    4454 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-089416bbf08c096de"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155144    4454 flags.go:64] FLAG: --qos-reserved=""
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155147    4454 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155150    4454 flags.go:64] FLAG: --register-node="true"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155153    4454 flags.go:64] FLAG: --register-with-taints=""
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155157    4454 flags.go:64] FLAG: --registry-burst="10"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155160    4454 flags.go:64] FLAG: --registry-qps="5"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155163    4454 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155166    4454 flags.go:64] FLAG: --reserved-memory=""
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155174    4454 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 21:29:51.163662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155177    4454 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155180    4454 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155183    4454 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155185    4454 flags.go:64] FLAG: --runonce="false"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155188    4454 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155191    4454 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155194    4454 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155197    4454 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155200    4454 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155202    4454 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155205    4454 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155208    4454 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155213    4454 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155216    4454 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155219    4454 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155222    4454 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155225    4454 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155228    4454 flags.go:64] FLAG: --system-cgroups=""
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155231    4454 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155236    4454 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155238    4454 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155241    4454 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155246    4454 flags.go:64] FLAG: --tls-min-version=""
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155249    4454 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155251    4454 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 21:29:51.164277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155254    4454 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155257    4454 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155260    4454 flags.go:64] FLAG: --v="2"
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155264    4454 flags.go:64] FLAG: --version="false"
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155267    4454 flags.go:64] FLAG: --vmodule=""
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155271    4454 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155275    4454 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155423    4454 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155428    4454 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155432    4454 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155435    4454 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155438    4454 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155442    4454 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155446    4454 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155448    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155451    4454 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155454    4454 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155456    4454 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155459    4454 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155465    4454 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155469    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:51.165085 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155473    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155477    4454 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155481    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155484    4454 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155486    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155489    4454 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155492    4454 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155494    4454 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155497    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155499    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155502    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155518    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155522    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155526    4454 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155529    4454 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155532    4454 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155535    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155537    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155541    4454 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:51.165960 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155543    4454 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155546    4454 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155548    4454 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155551    4454 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155553    4454 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155556    4454 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155558    4454 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155561    4454 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155563    4454 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155566    4454 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155568    4454 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155572    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155575    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155577    4454 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155580    4454 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155583    4454 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155585    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155588    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155590    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155593    4454 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:51.166498 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155595    4454 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155598    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155600    4454 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155602    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155605    4454 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155607    4454 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155610    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155612    4454 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155615    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155617    4454 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155620    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155622    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155625    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155627    4454 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155630    4454 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155632    4454 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155635    4454 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155637    4454 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155640    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155644    4454 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:51.167070 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155647    4454 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155650    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155652    4454 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155656    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155659    4454 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155661    4454 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155664    4454 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155666    4454 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155669    4454 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155671    4454 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155674    4454 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155676    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155679    4454 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155681    4454 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155684    4454 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155687    4454 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155689    4454 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155692    4454 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155696    4454 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:51.167575 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155699    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155702    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155705    4454 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155707    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155710    4454 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155712    4454 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155715    4454 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155717    4454 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155720    4454 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155722    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155725    4454 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155727    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155731    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155734    4454 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.155761    4454 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155794    4454 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:51.168122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155800    4454 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155803    4454 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155806    4454 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155809    4454 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155811    4454 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155814    4454 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155816    4454 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155819    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155821    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155824    4454 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155826    4454 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155829    4454 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155832    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155834    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155837    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155839    4454 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155841    4454 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155844    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155846    4454 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155850    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:51.168650 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155852    4454 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155854    4454 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155857    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155860    4454 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155863    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155865    4454 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155868    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155870    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155873    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155877    4454 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155880    4454 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155883    4454 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155887    4454 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155889    4454 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155892    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155894    4454 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155897    4454 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155899    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155902    4454 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:51.169223 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155904    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155907    4454 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155910    4454 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155912    4454 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155914    4454 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155917    4454 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155919    4454 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155922    4454 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155925    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155927    4454 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155930    4454 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155932    4454 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155935    4454 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155937    4454 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155939    4454 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155943    4454 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155947    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155964    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155969    4454 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:51.169836 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155972    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155974    4454 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155977    4454 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155981    4454 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155984    4454 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155986    4454 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155990    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155993    4454 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155996    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.155998    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156001    4454 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156003    4454 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156006    4454 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156009    4454 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156012    4454 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156015    4454 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156018    4454 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156022    4454 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156025    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:51.170304 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156028    4454 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156031    4454 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156033    4454 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156036    4454 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156038    4454 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156041    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156044    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156046    4454 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156049    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156052    4454 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156054    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156057    4454 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156060    4454 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156062    4454 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156065    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156068    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156072    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156074    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156076    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156081    4454 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:51.170791 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156084    4454 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156087    4454 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156090    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156092    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156095    4454 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156097    4454 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156099    4454 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.156102    4454 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.160149    4454 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.160170    4454 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.160194    4454 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.160202    4454 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160361    4454 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160369    4454 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160374    4454 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160378    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160383    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160388    4454 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160397    4454 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:51.171262 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160402    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160406    4454 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160411    4454 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160415    4454 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160419    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160423    4454 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160428    4454 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160433    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160437    4454 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160441    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160445    4454 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160450    4454 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160459    4454 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160463    4454 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160468    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160472    4454 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160477    4454 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160481    4454 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160486    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160492    4454 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:51.171771 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160498    4454 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160503    4454 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160529    4454 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160534    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160543    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160548    4454 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160552    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160558    4454 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160562    4454 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160570    4454 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160575    4454 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160579    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160584    4454 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160588    4454 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160593    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160598    4454 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160602    4454 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160628    4454 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160653    4454 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160657    4454 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:51.172250 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160789    4454 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160796    4454 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160801    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160806    4454 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160809    4454 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160814    4454 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160819    4454 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160822    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160825    4454 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160828    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160830    4454 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160833    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160836    4454 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160838    4454 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160841    4454 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160844    4454 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160846    4454 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160849    4454 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160852    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160856    4454 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:51.172732 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160858    4454 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160861    4454 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160864    4454 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160866    4454 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160869    4454 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160872    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160875    4454 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160877    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160880    4454 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160883    4454 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160886    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160888    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160891    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160895    4454 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160898    4454 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160901    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160903    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160905    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160908    4454 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:51.173219 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160911    4454 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160913    4454 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160916    4454 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160918    4454 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160921    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160924    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160926    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160929    4454 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160931    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160935    4454 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160937    4454 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160940    4454 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160944    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160947    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160949    4454 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160952    4454 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160954    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160957    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160960    4454 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:51.173724 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.160962    4454 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.160994    4454 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161142    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161147    4454 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161150    4454 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161153    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161156    4454 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161158    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161161    4454 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161163    4454 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161166    4454 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161169    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161171    4454 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161174    4454 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161177    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161179    4454 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:51.174180 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161182    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161184    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161187    4454 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161189    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161192    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161194    4454 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161197    4454 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161200    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161202    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161205    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161208    4454 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161211    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161214    4454 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161216    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161219    4454 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161221    4454 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161224    4454 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161227    4454 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161230    4454 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161232    4454 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:51.174584 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161235    4454 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161237    4454 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161240    4454 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161242    4454 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161245    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161248    4454 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161251    4454 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161253    4454 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161256    4454 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161259    4454 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161261    4454 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161264    4454 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161266    4454 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161269    4454 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161271    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161274    4454 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161276    4454 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161280    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161283    4454 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161285    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:51.175104 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161288    4454 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161291    4454 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161293    4454 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161296    4454 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161299    4454 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161301    4454 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161304    4454 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161306    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161309    4454 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161312    4454 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161316    4454 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161319    4454 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161322    4454 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161325    4454 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161328    4454 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161330    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161333    4454 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161336    4454 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161339    4454 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161341    4454 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:51.175910 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161344    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161346    4454 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161349    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161351    4454 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161354    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161357    4454 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161359    4454 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161362    4454 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161364    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161367    4454 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161370    4454 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161372    4454 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161375    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161377    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161380    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161383    4454 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161386    4454 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161390    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161393    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:51.176467 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161395    4454 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161398    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161400    4454 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161402    4454 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161405    4454 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161407    4454 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161410    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161412    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161415    4454 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161417    4454 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161420    4454 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161423    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161426    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.161448    4454 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161478    4454 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 21:29:51.177020 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161482    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161485    4454 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161488    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161491    4454 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161494    4454 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161497    4454 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161500    4454 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161502    4454 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161523    4454 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161526    4454 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161529    4454 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161532    4454 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161534    4454 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161537    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161540    4454 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161542    4454 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161545    4454 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161548    4454 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161550    4454 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161553    4454 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 21:29:51.177369 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161555    4454 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161558    4454 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161561    4454 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161563    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161566    4454 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161568    4454 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161571    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161574    4454 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161576    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161579    4454 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161582    4454 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161585    4454 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161587    4454 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161590    4454 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161593    4454 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161595    4454 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161598    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161600    4454 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161603    4454 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 21:29:51.177884 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161607    4454 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161611    4454 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161615    4454 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161618    4454 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161621    4454 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161623    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161627    4454 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161629    4454 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161632    4454 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161636    4454 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161638    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161641    4454 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161643    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161646    4454 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161648    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161651    4454 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161653    4454 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161656    4454 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161658    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161661    4454 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 21:29:51.178399 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161663    4454 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161665    4454 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161668    4454 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161671    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161673    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161680    4454 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161683    4454 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161687    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161690    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161693    4454 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161696    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161698    4454 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161700    4454 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161703    4454 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161706    4454 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161708    4454 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161711    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161713    4454 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161716    4454 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 21:29:51.179173 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161719    4454 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161721    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161724    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161726    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161729    4454 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161732    4454 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161734    4454 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161737    4454 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161739    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161742    4454 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161744    4454 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161747    4454 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161749    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161752    4454 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161754    4454 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161756    4454 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161759    4454 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161762    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161766    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 21:29:51.180046 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161768    4454 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161771    4454 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161773    4454 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161776    4454 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161778    4454 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161781    4454 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161784    4454 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:51.161787    4454 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.161844    4454 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.162792    4454 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.162843    4454 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.163016    4454 server.go:1014] "Starting client certificate rotation"
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.163168    4454 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.163228    4454 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 21:24:11 +0000 UTC" deadline="2028-06-27 08:46:05.230465335 +0000 UTC"
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.163273    4454 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14435h16m14.067195881s"
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.163552    4454 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 21:29:51.180919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.164592    4454 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 21:29:51.181623 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.166308    4454 log.go:25] "Validated CRI v1 runtime API"
Nov 03 21:29:51.181623 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.167723    4454 log.go:25] "Validated CRI v1 image API"
Nov 03 21:29:51.181623 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.168472    4454 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 21:29:51.181623 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.169926    4454 fs.go:135] Filesystem UUIDs: map[473c0aac-8d1d-4bc8-9723-fdedc9c6658f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 96804c93-4b39-43f0-ab87-d4dbce5a3177:/dev/nvme0n1p3]
Nov 03 21:29:51.181811 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.169945    4454 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/03f2395f5e0075fcc939aa1646b0cb4da91a99c2815ed1b182b4ce31dd7107d0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/03f2395f5e0075fcc939aa1646b0cb4da91a99c2815ed1b182b4ce31dd7107d0/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/155e9bae8c395f1c283b407f9e61547fa8dfd5178e22a3ad8c814f28fa8061c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/155e9bae8c395f1c283b407f9e61547fa8dfd5178e22a3ad8c814f28fa8061c5/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/454db50f9c3353a5d5e1ab583eb65f3ef2196c37ce2190dc7677978a33323636/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/454db50f9c3353a5d5e1ab583eb65f3ef2196c37ce2190dc7677978a33323636/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/47fd279add685d3efc6605df15dd287721c3f8b3b7ac04c68a1010dff9858590/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/47fd279add685d3efc6605df15dd287721c3f8b3b7ac04c68a1010dff9858590/userdata/shm major:0 minor:203 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6881e91a5d0f2a64c8329625738daf8aa57fa67891e0c2f31e3afb9d3aca3e73/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6881e91a5d0f2a64c8329625738daf8aa57fa67891e0c2f31e3afb9d3aca3e73/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/73cdd256c5d66f3cd132901d0cc1e43b746c929efed37dc7ff4b9abedd10a24e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/73cdd256c5d66f3cd132901d0cc1e43b746c929efed37dc7ff4b9abedd10a24e/userdata/shm major:0 minor:220 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7cd88de0cca83cd280cedb54ec74f8fc1e88469fbb76f7999c8e8d74bc84bdce/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7cd88de0cca83cd280cedb54ec74f8fc1e88469fbb76f7999c8e8d74bc84bdce/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/82699a23ce8e807ce50d3091f372547c6efdc8a8b97b83844034c16c23e67409/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/82699a23ce8e807ce50d3091f372547c6efdc8a8b97b83844034c16c23e67409/userdata/shm major:0 minor:218 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b3ea48873e2ce64601b2cb0af1686e3d7a0231bb0c3f3a61a6c5a05947a9fddf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b3ea48873e2ce64601b2cb0af1686e3d7a0231bb0c3f3a61a6c5a05947a9fddf/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c5320a9e6eb91b05defae8ce80631832933ef7a7280f7ede24826d59c4f0e0b6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c5320a9e6eb91b05defae8ce80631832933ef7a7280f7ede24826d59c4f0e0b6/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dd13bc15ca19c42bc46373cc3e0da80e0b5d90e2184c748cfea0b9bf9c6c2938/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dd13bc15ca19c42bc46373cc3e0da80e0b5d90e2184c748cfea0b9bf9c6c2938/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e09369fd42372eda0ccb8575fde6776bdcac58f8c9953cff6104442774ef77c4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e09369fd42372eda0ccb8575fde6776bdcac58f8c9953cff6104442774ef77c4/userdata/shm major:0 minor:216 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e92bf21eb5fcd6539d298e25967b983601226c1e20485c9dbc9201a88fa37f8f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e92bf21eb5fcd6539d298e25967b983601226c1e20485c9dbc9201a88fa37f8f/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fb1b078a90cb0432fc2929c8bf26878a7811f4ff102efd3c361e6deb32e3cbd0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fb1b078a90cb0432fc2929c8bf26878a7811f4ff102efd3c361e6deb32e3cbd0/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70/volumes/kubernetes.io~projected/kube-api-access-82wfl:{mountpoint:/var/lib/kubelet/pods/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70/volumes/kubernetes.io~projected/kube-api-access-82wfl major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/15567c09-c50f-4c3b-81b5-5357f08fa543/volumes/kubernetes.io~projected/kube-api-access-tctvw:{mountpoint:/var/lib/kubelet/pods/15567c09-c50f-4c3b-81b5-5357f08fa543/volumes/kubernetes.io~projected/kube-api-access-tctvw major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/15567c09-c50f-4c3b-81b5-5357f08fa543/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/15567c09-c50f-4c3b-81b5-5357f08fa543/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/29ad1278-e154-41fb-8648-0adc8d7a648f/volumes/kubernetes.io~projected/kube-api-access-2fh5b:{mountpoint:/var/lib/kubelet/pods/29ad1278-e154-41fb-8648-0adc8d7a648f/volumes/kubernetes.io~projected/kube-api-access-2fh5b major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3a287e20-e2d0-451a-97ce-b4fddffeb788/volumes/kubernetes.io~projected/kube-api-access-gkpsd:{mountpoint:/var/lib/kubelet/pods/3a287e20-e2d0-451a-97ce-b4fddffeb788/volumes/kubernetes.io~projected/kube-api-access-gkpsd major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/45263f32-066f-4bdf-84b9-a3ebd1f33f90/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/45263f32-066f-4bdf-84b9-a3ebd1f33f90/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/47bb3d82-52e3-4e39-8b1a-1deb886a29db/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/47bb3d82-52e3-4e39-8b1a-1deb886a29db/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/47bb3d82-52e3-4e39-8b1a-1deb886a29db/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/47bb3d82-52e3-4e39-8b1a-1deb886a29db/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/47bb3d82-52e3-4e39-8b1a-1deb886a29db/volumes/kubernetes.io~projected/kube-api-access-7tfpz:{mountpoint:/var/lib/kubelet/pods/47bb3d82-52e3-4e39-8b1a-1deb886a29db/volumes/kubernetes.io~projected/kube-api-access-7tfpz major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8bcb1ef0-aa94-4932-99a6-53178e770a68/volumes/kubernetes.io~projected/kube-api-access-bvfs2:{mountpoint:/var/lib/kubelet/pods/8bcb1ef0-aa94-4932-99a6-53178e770a68/volumes/kubernetes.io~projected/kube-api-access-bvfs2 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa5882bd-1e86-428d-afec-490ebd2da878/volumes/kubernetes.io~projected/kube-api-access-6lz9n:{mountpoint:/var/lib/kubelet/pods/aa5882bd-1e86-428d-afec-490ebd2da878/volumes/kubernetes.io~projected/kube-api-access-6lz9n major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/af4170fc-093a-4066-bf79-4f57388f073d/volumes/kubernetes.io~projected/kube-api-access-jlk69:{mountpoint:/var/lib/kubelet/pods/af4170fc-093a-4066-bf79-4f57388f073d/volumes/kubernetes.io~projected/kube-api-access-jlk69 major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b7e2c320-fbf4-40e6-938d-b4e1377d769c/volumes/kubernetes.io~projected/kube-api-access-dq4wr:{mountpoint:/var/lib/kubelet/pods/b7e2c320-fbf4-40e6-938d-b4e1377d769c/volumes/kubernetes.io~projected/kube-api-access-dq4wr major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b978b73d-666a-4c25-b959-74d3e14c7bbf/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/b978b73d-666a-4c25-b959-74d3e14c7bbf/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a/volumes/kubernetes.io~projected/kube-api-access-dx8fk:{mountpoint:/var/lib/kubelet/pods/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a/volumes/kubernetes.io~projected/kube-api-access-dx8fk major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a/volumes/kubernetes.io~secret/metrics-certs major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d09b08fd-64c6-4b9e-ba06-273cbedbd58a/volumes/kubernetes.io~projected/kube-api-access-6qcc8:{mountpoint:/var/lib/kubelet/pods/d09b08fd-64c6-4b9e-ba06-273cbedbd58a/volumes/kubernetes.io~projected/kube-api-access-6qcc8 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe26d35e-47a4-497a-9fe1-64cc807267e9/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/fe26d35e-47a4-497a-9fe1-64cc807267e9/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe26d35e-47a4-497a-9fe1-64cc807267e9/volumes/kubernetes.io~projected/kube-api-access-2gfqr:{mountpoint:/var/lib/kubelet/pods/fe26d35e-47a4-497a-9fe1-64cc807267e9/volumes/kubernetes.io~projected/kube-api-access-2gfqr major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe26d35e-47a4-497a-9fe1-64cc807267e9/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/fe26d35e-47a4-497a-9fe1-64cc807267e9/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b/volumes/kubernetes.io~projected/kube-api-access-jgc4m:{mountpoint:/var/lib/kubelet/pods/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b/volumes/kubernetes.io~projected/kube-api-access-jgc4m major:0 minor:202 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/34b961de324bff1d5a2dac8eecedfbf76ed20cc29f50ecf890faaefa64113f5f/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/5e5f57639066ab9ec709df672e9216485cbdc60b6cdf645abcb7b267e7b46681/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/14e110dcf547f8734fe9d22a5ddb030663fdc2a5b8107dac2c46327e1e2c1109/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/519e99b4ba655bc4bc46538682fec1fddac3c5a133005b4cd8ceb5c85c560f6c/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/ca521621db3931b89de010c0dfc8717263592e546ed7702072914279400ba468/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/bcfd58d506b6b77b1d8f5b76d81b9054ebd81eb85dd3cad899305bc3ec8f9b5a/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/f926607610a00e9b26e94d0e74065afe88ad9f9fcb166d11991ce266a565459f/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/760dddbc4b1b4915e07d371a677879d8a573abeedbb3539c09193e21dcde4233/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/e1406b12b2c084835cb29bf620cec6a3c3452ff3625d603d7bbcbddb92274f14/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/92fcc74ab6c6c3e69e1d29093440ba7bcbe3d9c2d0ae6def453d6eaa2d86b579/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/4a8d5cd70a16ae14983502966e1302bd93728c5d6d96765f30843bc048013330/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/7f34899eed04ffea4066ba2a34f0290d5760f8012b8a8d1c566823c251305e4c/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/948da47a5071b18339c6df638623395fcbb7dc5b0854177a9344931e335abb4b/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-153:{mountpoint:/var/lib/containers/storage/overlay/1bbc149ad52c13f786b1bb43c2d41b5490c7cdc5b6ca6c0a8451d094c806de56/merged major:0 minor:153 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/a929583d07ad80244704765fe0cdce23496ce674d1c20bdaf803778335d7eb0b/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/b3025a7a0ecf5658bef273b1aef9e3ba0c658695d6333ef2ce8cf317ecd1232c/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/918e613b727eaa89ba47ad782fec20627c667041782a1db3ef5dbb33fa9d8236/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/e9033c7b9f1b06b22022b74a272afd3c3488da1e9626801aeec895961d5c34e9/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/5b0f0820a77d55620950c176471532600744390bde8139f1080f0c6241ed84b3/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/f1be6bbf3e2dae8597b44f296c20e05629efe2be8ea57b3e882c146c6483d942/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/b1529832ef9034f5d6c3a32a36784f4e8e4372e978144fcc1dc0bbcbc25aabe0/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-205:{mountpoint:/var/lib/containers/storage/overlay/72f88ee66eeb6ed07b98016008fe6d6c3d5bee98ed69299964428304e4bc7ee0/merged major:0 minor:205 fsType:overlay blockSize:0} overlay_0-207:{mountpoint:/var/lib/containers/storage/overlay/8c5b7b936d83964a3b086bd3d8d0d6c123616acdc402f934f4b8c7686e82f53e/merged major:0 minor:207 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/b91550ff15d78c4881b09cf7beba469528985bdd5406163438769bc4ca562b83/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-224:{mountpoint:/var/lib/containers/storage/overlay/7683a894402ea8b9e749dc895b04538e2b2e8398cc6206a0271306ac18728246/merged major:0 minor:224 fsType:overlay blockSize:0} overlay_0-226:{mountpoint:/var/lib/containers/storage/overlay/c134cd9686ca7d76176390d2813c31e42e96c40034682d58b58b9597cfe82347/merged major:0 minor:226 fsType:overlay blockSize:0} overlay_0-228:{mountpoint:/var/lib/containers/storage/overlay/32f882c84a325918989616809e5ebf173abf1163a06b0fd44f2d3397ec6e9d91/merged major:0 minor:228 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/0722841ced6e749632203537d8fdb446dd4e6ed0ce2922b5a21dbef697a64d63/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/6248f5a7b6f07de9ee3d32de4eeb81d60e1e20af27ba2c3d0a1d18a36687c748/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/781447cab1e6d117d98747726977c86d5b5c958beff5b56c80a57721fcf2494c/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/5f4c106cb93ef5d52fad5e316562fba7ce5e12b210e14485d28e12594e286aa7/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/598c982d5da7dc50753c75f86e6e682a9633e74befe10283b9f50e438bd96eab/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/b03be6c3e3967810e8148534aacca4a404d74ee350cf1a77dc8a89e078c4c628/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/e0dd1b262bb57002f2188fb7f3940112331574dfafab48e283aa867cb439d4f2/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/9a69093623f08c939e886380dfa87c1abb933a993c61b8677c33c8d7255f94e4/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/45b82c93622d8c2409dbb342e2ccc44fe2c39303835a24a3eda764748038f872/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/63a9bde0631882411282b0f0707f60558416d17c5c22119d2f87870c9d102fa8/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/9fd7190f0a70be39abbaf334130d6455daff23082265bc7eb9fc92889e48ba8b/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/c844df926edb81727ab365fb009ab6ff4eb8910e65e9f9f17d97c0521ee1c06b/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/29a946fb3e41c10a463def60e37164a22af0ca42691bd6f040ff04a839d7d8a0/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.175930    4454 manager.go:217] Machine: {Timestamp:2026-11-03 21:29:51.17514775 +0000 UTC m=+0.053970026 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200183 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec246965302b872c5175edbeb1aa6489 SystemUUID:ec246965-302b-872c-5175-edbeb1aa6489 BootID:6df89aeb-1239-4247-9deb-8bea5ff07a7b Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/3a287e20-e2d0-451a-97ce-b4fddffeb788/volumes/kubernetes.io~projected/kube-api-access-gkpsd DeviceMajor:0 DeviceMinor:64 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/03f2395f5e0075fcc939aa1646b0cb4da91a99c2815ed1b182b4ce31dd7107d0/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b/volumes/kubernetes.io~projected/kube-api-access-jgc4m DeviceMajor:0 DeviceMinor:202 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/47bb3d82-52e3-4e39-8b1a-1deb886a29db/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:58 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/47bb3d82-52e3-4e39-8b1a-1deb886a29db/volumes/kubernetes.io~projected/kube-api-access-7tfpz DeviceMajor:0 DeviceMinor:60 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/fe26d35e-47a4-497a-9fe1-64cc807267e9/volumes/kubernetes.io~projected/kube-api-access-2gfqr DeviceMajor:0 DeviceMinor:63 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7cd88de0cca83cd280cedb54ec74f8fc1e88469fbb76f7999c8e8d74bc84bdce/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b978b73d-666a-4c25-b959-74d3e14c7bbf/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:214 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dd13bc15ca19c42bc46373cc3e0da80e0b5d90e2184c748cfea0b9bf9c6c2938/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/454db50f9c3353a5d5e1ab583eb65f3ef2196c37ce2190dc7677978a33323636/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a/volumes/kubernetes.io~projected/kube-api-access-dx8fk DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/af4170fc-093a-4066-bf79-4f57388f073d/volumes/kubernetes.io~projected/kube-api-access-jlk69 DeviceMajor:0 DeviceMinor:61 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aa5882bd-1e86-428d-afec-490ebd2da878/volumes/kubernetes.io~projected/kube-api-access-6lz9n DeviceMajor:0 DeviceMinor:65 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6881e91a5d0f2a64c8329625738daf8aa57fa67891e0c2f31e3afb9d3aca3e73/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/45263f32-066f-4bdf-84b9-a3ebd1f33f90/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/b7e2c320-fbf4-40e6-938d-b4e1377d769c/volumes/kubernetes.io~projected/kube-api-access-dq4wr DeviceMajor:0 DeviceMinor:62 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/8bcb1ef0-aa94-4932-99a6-53178e770a68/volumes/kubernetes.io~projected/kube-api-access-bvfs2 DeviceMajor:0 DeviceMinor:67 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/29ad1278-e154-41fb-8648-0adc8d7a648f/volumes/kubernetes.io~projected/kube-api-access-2fh5b DeviceMajor:0 DeviceMinor:215 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-228 DeviceMajor:0 DeviceMinor:228 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/155e9bae8c395f1c283b407f9e61547fa8dfd5178e22a3ad8c814f28fa8061c5/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fb1b078a90cb0432fc2929c8bf26878a7811f4ff102efd3c361e6deb32e3cbd0/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e92bf21eb5fcd6539d298e25967b983601226c1e20485c9dbc9201a88fa37f8f/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-153 DeviceMajor:0 DeviceMinor:153 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/82699a23ce8e807ce50d3091f372547c6efdc8a8b97b83844034c16c23e67409/userdata/shm DeviceMajor:0 DeviceMinor:218 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-226 DeviceMajor:0 DeviceMinor:226 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70/volumes/kubernetes.io~projected/kube-api-access-82wfl DeviceMajor:0 DeviceMinor:66 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b3ea48873e2ce64601b2cb0af1686e3d7a0231bb0c3f3a61a6c5a05947a9fddf/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-205 DeviceMajor:0 DeviceMinor:205 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d09b08fd-64c6-4b9e-ba06-273cbedbd58a/volumes/kubernetes.io~projected/kube-api-access-6qcc8 DeviceMajor:0 DeviceMinor:201 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/47fd279add685d3efc6605df15dd287721c3f8b3b7ac04c68a1010dff9858590/userdata/shm DeviceMajor:0 DeviceMinor:203 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fe26d35e-47a4-497a-9fe1-64cc807267e9/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-207 DeviceMajor:0 DeviceMinor:207 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:209 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e09369fd42372eda0ccb8575fde6776bdcac58f8c9953cff6104442774ef77c4/userdata/shm DeviceMajor:0 DeviceMinor:216 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/73cdd256c5d66f3cd132901d0cc1e43b746c929efed37dc7ff4b9abedd10a24e/userdata/shm DeviceMajor:0 DeviceMinor:220 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/fe26d35e-47a4-497a-9fe1-64cc807267e9/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/15567c09-c50f-4c3b-81b5-5357f08fa543/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-224 DeviceMajor:0 DeviceMinor:224 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c5320a9e6eb91b05defae8ce80631832933ef7a7280f7ede24826d59c4f0e0b6/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/47bb3d82-52e3-4e39-8b1a-1deb886a29db/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/15567c09-c50f-4c3b-81b5-5357f08fa543/volumes/kubernetes.io~projected/kube-api-access-tctvw DeviceMajor:0 DeviceMinor:197 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:47fd279add685d3 MacAddress:d2:c3:c0:e3:39:85 Speed:10000 Mtu:8901} {Name:73cdd256c5d66f3 MacAddress:86:84:84:e5:3a:d9 Speed:10000 Mtu:8901} {Name:82699a23ce8e807 MacAddress:46:32:ad:5f:6c:35 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:c9:10:1d:4f Speed:0 Mtu:9001} {Name:br-int MacAddress:12:d5:d6:dd:cb:23 Speed:0 Mtu:8901} {Name:e09369fd42372ed MacAddress:ca:be:fb:c9:90:d1 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:c9:10:1d:4f Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:4e:63:36:50:55:40 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:16:f5:d9:c1:8a:bb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.176587    4454 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.176644    4454 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.176931    4454 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.176950    4454 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-212.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.177066    4454 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.177074    4454 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.177084    4454 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.177096    4454 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.177172    4454 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.177326    4454 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.177426    4454 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.177460    4454 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.177475    4454 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.177485    4454 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.177493    4454 kubelet.go:405] "Adding apiserver pod source"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.177501    4454 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.178208    4454 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.178221    4454 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.178722    4454 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.181351    4454 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.181363    4454 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.181480    4454 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.181751    4454 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.181795    4454 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182001    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182019    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182028    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182037    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 21:29:51.182031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182051    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 21:29:51.183568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182060    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 21:29:51.183568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182071    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 21:29:51.183568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182080    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 21:29:51.183568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182091    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 21:29:51.183568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182100    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 21:29:51.183568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182118    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 21:29:51.183568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182132    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 21:29:51.183568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182166    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 21:29:51.183568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.182176    4454 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 21:29:51.185825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.185809    4454 server.go:1263] "Started kubelet"
Nov 03 21:29:51.185917 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.185874    4454 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 21:29:51.185982 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.185940    4454 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 21:29:51.186031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.186012    4454 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 21:29:51.186275 ip-10-0-137-212 systemd[1]: Started Kubernetes Kubelet.
Nov 03 21:29:51.186457 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.186351    4454 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 21:29:51.187031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.187015    4454 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 21:29:51.188595 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.188579    4454 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:29:51.188734    4454 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.188773    4454 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.188782    4454 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.188958    4454 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.189275    4454 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.189324    4454 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 21:24:13 +0000 UTC" deadline="2028-04-27 01:49:47.290801269 +0000 UTC"
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.189356    4454 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12964h19m56.101448539s"
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190081    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7e2c320-fbf4-40e6-938d-b4e1377d769c" volumeName="kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190109    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15567c09-c50f-4c3b-81b5-5357f08fa543" volumeName="kubernetes.io/secret/15567c09-c50f-4c3b-81b5-5357f08fa543-cert" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190123    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3a287e20-e2d0-451a-97ce-b4fddffeb788" volumeName="kubernetes.io/configmap/3a287e20-e2d0-451a-97ce-b4fddffeb788-serviceca" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190135    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3a287e20-e2d0-451a-97ce-b4fddffeb788" volumeName="kubernetes.io/projected/3a287e20-e2d0-451a-97ce-b4fddffeb788-kube-api-access-gkpsd" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190148    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="45263f32-066f-4bdf-84b9-a3ebd1f33f90" volumeName="kubernetes.io/configmap/45263f32-066f-4bdf-84b9-a3ebd1f33f90-konnectivity-ca" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190160    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="45263f32-066f-4bdf-84b9-a3ebd1f33f90" volumeName="kubernetes.io/secret/45263f32-066f-4bdf-84b9-a3ebd1f33f90-agent-certs" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190172    4454 factory.go:55] Registering systemd factory
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190185    4454 factory.go:223] Registration of the systemd container factory successfully
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190170    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bcb1ef0-aa94-4932-99a6-53178e770a68" volumeName="kubernetes.io/configmap/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-daemon-config" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190259    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af4170fc-093a-4066-bf79-4f57388f073d" volumeName="kubernetes.io/configmap/af4170fc-093a-4066-bf79-4f57388f073d-iptables-alerter-script" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190278    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a" volumeName="kubernetes.io/projected/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-kube-api-access-dx8fk" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190290    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="29ad1278-e154-41fb-8648-0adc8d7a648f" volumeName="kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190316    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47bb3d82-52e3-4e39-8b1a-1deb886a29db" volumeName="kubernetes.io/empty-dir/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-tuned" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190329    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bcb1ef0-aa94-4932-99a6-53178e770a68" volumeName="kubernetes.io/projected/8bcb1ef0-aa94-4932-99a6-53178e770a68-kube-api-access-bvfs2" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190340    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe26d35e-47a4-497a-9fe1-64cc807267e9" volumeName="kubernetes.io/projected/fe26d35e-47a4-497a-9fe1-64cc807267e9-kube-api-access-2gfqr" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190352    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7e2c320-fbf4-40e6-938d-b4e1377d769c" volumeName="kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 03 21:29:51.190393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.190367    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d09b08fd-64c6-4b9e-ba06-273cbedbd58a" volumeName="kubernetes.io/configmap/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-config-volume" seLinuxMountContext=""
Nov 03 21:29:51.193568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193540    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d09b08fd-64c6-4b9e-ba06-273cbedbd58a" volumeName="kubernetes.io/projected/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-kube-api-access-6qcc8" seLinuxMountContext=""
Nov 03 21:29:51.193656 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193575    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7e2c320-fbf4-40e6-938d-b4e1377d769c" volumeName="kubernetes.io/projected/b7e2c320-fbf4-40e6-938d-b4e1377d769c-kube-api-access-dq4wr" seLinuxMountContext=""
Nov 03 21:29:51.193656 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193598    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d09b08fd-64c6-4b9e-ba06-273cbedbd58a" volumeName="kubernetes.io/empty-dir/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-tmp-dir" seLinuxMountContext=""
Nov 03 21:29:51.193656 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193615    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe26d35e-47a4-497a-9fe1-64cc807267e9" volumeName="kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovnkube-config" seLinuxMountContext=""
Nov 03 21:29:51.193656 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193629    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe26d35e-47a4-497a-9fe1-64cc807267e9" volumeName="kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-env-overrides" seLinuxMountContext=""
Nov 03 21:29:51.193656 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193647    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe26d35e-47a4-497a-9fe1-64cc807267e9" volumeName="kubernetes.io/secret/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 03 21:29:51.193857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193660    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9199eb-c6a1-4a45-9b7b-862b6e778f4b" volumeName="kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 03 21:29:51.193857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193678    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0fe02b31-bc0e-4ffc-916d-f6cd60b68a70" volumeName="kubernetes.io/empty-dir/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-tmp-dir" seLinuxMountContext=""
Nov 03 21:29:51.193857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193691    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8bcb1ef0-aa94-4932-99a6-53178e770a68" volumeName="kubernetes.io/configmap/8bcb1ef0-aa94-4932-99a6-53178e770a68-cni-binary-copy" seLinuxMountContext=""
Nov 03 21:29:51.193857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193704    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe26d35e-47a4-497a-9fe1-64cc807267e9" volumeName="kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovnkube-script-lib" seLinuxMountContext=""
Nov 03 21:29:51.193857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193727    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9199eb-c6a1-4a45-9b7b-862b6e778f4b" volumeName="kubernetes.io/projected/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-api-access-jgc4m" seLinuxMountContext=""
Nov 03 21:29:51.193857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193742    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a" volumeName="kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs" seLinuxMountContext=""
Nov 03 21:29:51.193857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193762    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0fe02b31-bc0e-4ffc-916d-f6cd60b68a70" volumeName="kubernetes.io/projected/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-kube-api-access-82wfl" seLinuxMountContext=""
Nov 03 21:29:51.193857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193793    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15567c09-c50f-4c3b-81b5-5357f08fa543" volumeName="kubernetes.io/projected/15567c09-c50f-4c3b-81b5-5357f08fa543-kube-api-access-tctvw" seLinuxMountContext=""
Nov 03 21:29:51.193857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193808    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47bb3d82-52e3-4e39-8b1a-1deb886a29db" volumeName="kubernetes.io/empty-dir/47bb3d82-52e3-4e39-8b1a-1deb886a29db-tmp" seLinuxMountContext=""
Nov 03 21:29:51.193857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193824    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47bb3d82-52e3-4e39-8b1a-1deb886a29db" volumeName="kubernetes.io/projected/47bb3d82-52e3-4e39-8b1a-1deb886a29db-kube-api-access-7tfpz" seLinuxMountContext=""
Nov 03 21:29:51.193857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193847    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa5882bd-1e86-428d-afec-490ebd2da878" volumeName="kubernetes.io/projected/aa5882bd-1e86-428d-afec-490ebd2da878-kube-api-access-6lz9n" seLinuxMountContext=""
Nov 03 21:29:51.194281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193861    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af4170fc-093a-4066-bf79-4f57388f073d" volumeName="kubernetes.io/projected/af4170fc-093a-4066-bf79-4f57388f073d-kube-api-access-jlk69" seLinuxMountContext=""
Nov 03 21:29:51.194281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193881    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b978b73d-666a-4c25-b959-74d3e14c7bbf" volumeName="kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret" seLinuxMountContext=""
Nov 03 21:29:51.194281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193896    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7e2c320-fbf4-40e6-938d-b4e1377d769c" volumeName="kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cni-binary-copy" seLinuxMountContext=""
Nov 03 21:29:51.194281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193910    4454 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff9199eb-c6a1-4a45-9b7b-862b6e778f4b" volumeName="kubernetes.io/empty-dir/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-data-volume" seLinuxMountContext=""
Nov 03 21:29:51.194281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193923    4454 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 21:29:51.194281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.193932    4454 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 21:29:51.197861 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.197701    4454 factory.go:153] Registering CRI-O factory
Nov 03 21:29:51.197861 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.197724    4454 factory.go:223] Registration of the crio container factory successfully
Nov 03 21:29:51.197861 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.197786    4454 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 21:29:51.197861 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.197808    4454 factory.go:103] Registering Raw factory
Nov 03 21:29:51.197861 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.197823    4454 manager.go:1199] Started watching for new ooms in manager
Nov 03 21:29:51.198949 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.198339    4454 manager.go:319] Starting recovery of all containers
Nov 03 21:29:51.205151 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.205133    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:29:51.205261 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.205242    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:29:51.221044 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.221004    4454 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 21:29:51.222304 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.222280    4454 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 21:29:51.222304 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.222303    4454 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 21:29:51.222453 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.222324    4454 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 21:29:51.222453 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:29:51.222365    4454 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 03 21:29:51.228982 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.228954    4454 generic.go:356] "Generic (PLEG): container finished" podID="86212ec82a64966b4f712c3fdb2b1cf2" containerID="1893520b5838a40c7096adede4496971154d47e96558f015dff62011774187b6" exitCode=0
Nov 03 21:29:51.234056 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.234036    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:29:51.234386 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.234364    4454 generic.go:356] "Generic (PLEG): container finished" podID="fe26d35e-47a4-497a-9fe1-64cc807267e9" containerID="8e26501d16b99372702bdec937ea4fb8522f8c991892eb6b00ac346ad622ec33" exitCode=1
Nov 03 21:29:51.234842 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.234824    4454 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 03 21:29:51.235093 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.234998    4454 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 21:29:51.242160 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.242134    4454 generic.go:356] "Generic (PLEG): container finished" podID="b7e2c320-fbf4-40e6-938d-b4e1377d769c" containerID="3561b6f98e60f401ecb00997ac36571471777fe063a86eb21e2d01b199cf20ff" exitCode=0
Nov 03 21:29:51.242230 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.242162    4454 generic.go:356] "Generic (PLEG): container finished" podID="b7e2c320-fbf4-40e6-938d-b4e1377d769c" containerID="4a4a93ccbc8e63a866961e4e83df497c408fc1867609942f0899607a01daab4c" exitCode=0
Nov 03 21:29:51.242230 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.242173    4454 generic.go:356] "Generic (PLEG): container finished" podID="b7e2c320-fbf4-40e6-938d-b4e1377d769c" containerID="4711f9d2a4cb168aea7099fb34e153503b9ecea8a82e92b8be9ad341fa039c86" exitCode=0
Nov 03 21:29:51.242230 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.242182    4454 generic.go:356] "Generic (PLEG): container finished" podID="b7e2c320-fbf4-40e6-938d-b4e1377d769c" containerID="7cb88288417ee31ee8816573830bc20ad8984152ee9950d1483030f7de66533a" exitCode=0
Nov 03 21:29:51.242230 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.242190    4454 generic.go:356] "Generic (PLEG): container finished" podID="b7e2c320-fbf4-40e6-938d-b4e1377d769c" containerID="301c335e7e75b5c49d42e611674255211f49cb22278bb08611aad53df345629e" exitCode=0
Nov 03 21:29:51.242230 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.242197    4454 generic.go:356] "Generic (PLEG): container finished" podID="b7e2c320-fbf4-40e6-938d-b4e1377d769c" containerID="a38cac1102562df23d22b196259b451a7dac03d5b07e9c16cd5e503c6a719472" exitCode=0
Nov 03 21:29:51.248749 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.248720    4454 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 21:29:51.248831 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.248754    4454 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 21:29:51.260905 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.260890    4454 manager.go:324] Recovery completed
Nov 03 21:29:51.274383 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.274359    4454 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 21:29:51.275610 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.275592    4454 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientMemory"
Nov 03 21:29:51.275689 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.275622    4454 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasNoDiskPressure"
Nov 03 21:29:51.275689 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.275633    4454 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientPID"
Nov 03 21:29:51.276928 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.276905    4454 cpu_manager.go:226] "Starting" policy="none"
Nov 03 21:29:51.276928 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.276918    4454 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 21:29:51.276928 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.276921    4454 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 21:29:51.276928 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.276934    4454 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 21:29:51.277153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277044    4454 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 03 21:29:51.277153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277057    4454 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 03 21:29:51.277153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277073    4454 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 03 21:29:51.277153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277078    4454 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 03 21:29:51.277153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277084    4454 policy_none.go:50] "Start"
Nov 03 21:29:51.277153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277095    4454 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 21:29:51.277153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277107    4454 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 21:29:51.277324 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277235    4454 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 03 21:29:51.277324 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277242    4454 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 03 21:29:51.277324 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277247    4454 policy_none.go:44] "Start"
Nov 03 21:29:51.277947 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277932    4454 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientMemory"
Nov 03 21:29:51.278022 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277961    4454 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasNoDiskPressure"
Nov 03 21:29:51.278022 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.277976    4454 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientPID"
Nov 03 21:29:51.281136 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.281108    4454 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 21:29:51.284218 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.284206    4454 manager.go:341] "Starting Device Plugin manager"
Nov 03 21:29:51.284898 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:29:51.284229    4454 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 21:29:51.284932 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.284905    4454 server.go:87] "Starting device plugin registration server"
Nov 03 21:29:51.285219 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.285208    4454 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 21:29:51.285247 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.285223    4454 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 21:29:51.285343 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.285326    4454 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 21:29:51.285969 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.285916    4454 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 21:29:51.286056 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.285978    4454 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 21:29:51.286056 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.285990    4454 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 21:29:51.286154 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.286061    4454 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T21:29:51.285965853Z","UUID":"444a17fa-609e-4827-b222-22c22fe1a515","Handler":null,"Name":"","Endpoint":""}
Nov 03 21:29:51.286642 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:29:51.286623    4454 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 21:29:51.286731 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:29:51.286655    4454 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-212.ec2.internal\" not found"
Nov 03 21:29:51.287400 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.287382    4454 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 21:29:51.287480 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.287405    4454 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 21:29:51.309968 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.309944    4454 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 21:29:51.322924 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.322889    4454 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"]
Nov 03 21:29:51.323315 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.323287    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal" event={"ID":"86212ec82a64966b4f712c3fdb2b1cf2","Type":"ContainerDied","Data":"1893520b5838a40c7096adede4496971154d47e96558f015dff62011774187b6"}
Nov 03 21:29:51.323409 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.323321    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal" event={"ID":"86212ec82a64966b4f712c3fdb2b1cf2","Type":"ContainerStarted","Data":"454db50f9c3353a5d5e1ab583eb65f3ef2196c37ce2190dc7677978a33323636"}
Nov 03 21:29:51.323409 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.323377    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal" event={"ID":"9caeb2eb053e15a50e91a5dd1ebbc6a1","Type":"ContainerStarted","Data":"ff2bb96756e497427a60d202bf6c9f784834093f26742caa64ac9d7ee0c57d76"}
Nov 03 21:29:51.323409 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.323390    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal" event={"ID":"9caeb2eb053e15a50e91a5dd1ebbc6a1","Type":"ContainerStarted","Data":"c5320a9e6eb91b05defae8ce80631832933ef7a7280f7ede24826d59c4f0e0b6"}
Nov 03 21:29:51.323533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.323430    4454 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.323533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.323496    4454 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.345123 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.345107    4454 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 21:29:51.345208 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:29:51.345142    4454 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-137-212.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.345570 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.345557    4454 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 21:29:51.345608 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:29:51.345589    4454 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.385787 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.385767    4454 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 21:29:51.387066 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.387052    4454 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientMemory"
Nov 03 21:29:51.387152 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.387079    4454 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasNoDiskPressure"
Nov 03 21:29:51.387152 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.387090    4454 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-212.ec2.internal" event="NodeHasSufficientPID"
Nov 03 21:29:51.387152 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.387125    4454 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.390271 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.390259    4454 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.409736 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.409724    4454 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 21:29:51.409784 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:29:51.409754    4454 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.409784 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.409765    4454 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.418368 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.418343    4454 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.418469 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.418404    4454 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.428511 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.428493    4454 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 21:29:51.428599 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:29:51.428557    4454 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-137-212.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.436331 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.436300    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/86212ec82a64966b4f712c3fdb2b1cf2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" (UID: \"86212ec82a64966b4f712c3fdb2b1cf2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.436422 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.436332    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86212ec82a64966b4f712c3fdb2b1cf2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" (UID: \"86212ec82a64966b4f712c3fdb2b1cf2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.436422 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.436356    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9caeb2eb053e15a50e91a5dd1ebbc6a1-config\") pod \"kube-apiserver-proxy-ip-10-0-137-212.ec2.internal\" (UID: \"9caeb2eb053e15a50e91a5dd1ebbc6a1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.537142 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.537110    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86212ec82a64966b4f712c3fdb2b1cf2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" (UID: \"86212ec82a64966b4f712c3fdb2b1cf2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.537286 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.537145    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9caeb2eb053e15a50e91a5dd1ebbc6a1-config\") pod \"kube-apiserver-proxy-ip-10-0-137-212.ec2.internal\" (UID: \"9caeb2eb053e15a50e91a5dd1ebbc6a1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.537286 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.537169    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/86212ec82a64966b4f712c3fdb2b1cf2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" (UID: \"86212ec82a64966b4f712c3fdb2b1cf2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.537286 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.537216    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/86212ec82a64966b4f712c3fdb2b1cf2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" (UID: \"86212ec82a64966b4f712c3fdb2b1cf2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.537286 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.537223    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9caeb2eb053e15a50e91a5dd1ebbc6a1-config\") pod \"kube-apiserver-proxy-ip-10-0-137-212.ec2.internal\" (UID: \"9caeb2eb053e15a50e91a5dd1ebbc6a1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:51.537286 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:51.537223    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86212ec82a64966b4f712c3fdb2b1cf2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" (UID: \"86212ec82a64966b4f712c3fdb2b1cf2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:52.178213 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.178180    4454 apiserver.go:52] "Watching apiserver"
Nov 03 21:29:52.228019 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.227987    4454 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 21:29:52.228422 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.228404    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal","openshift-multus/multus-additional-cni-plugins-swgh5","openshift-dns/dns-default-sz7tz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk","openshift-cluster-node-tuning-operator/tuned-x55l2","openshift-multus/multus-q2zcj","openshift-image-registry/node-ca-wjgz5","openshift-ovn-kubernetes/ovnkube-node-6xdtq","kube-system/global-pull-secret-syncer-vmr9k","openshift-ingress-canary/ingress-canary-kbh4v","openshift-dns/node-resolver-529hb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal","openshift-multus/network-metrics-daemon-4gsg9","openshift-network-diagnostics/network-check-target-7s62j","openshift-network-operator/iptables-alerter-wk2lw","kube-system/konnectivity-agent-79tpv","openshift-insights/insights-runtime-extractor-prbdm"]
Nov 03 21:29:52.232396 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.232375    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.232657 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.232634    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:52.245355 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.245337    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 21:29:52.245494 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.245403    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 21:29:52.245494 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.245434    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-rf9jq\""
Nov 03 21:29:52.245494 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.245461    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.246318    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.246534    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.246572    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.246662    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.246800    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.246858    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.246898    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.246947    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6tvtx\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.247001    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.247093    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.247105    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.247267    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-2kjq7\""
Nov 03 21:29:52.248501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.247319    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 21:29:52.256769 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.256751    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 21:29:52.256934 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.256870    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w4krc\""
Nov 03 21:29:52.256934 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.256913    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-xgwf4\""
Nov 03 21:29:52.256934 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.256927    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tnnmd\""
Nov 03 21:29:52.257129 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.256961    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 21:29:52.257129 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.256964    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-6tkv6\""
Nov 03 21:29:52.257129 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257063    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 21:29:52.257249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257134    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mhg7x\""
Nov 03 21:29:52.257249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257176    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-d5mwz\""
Nov 03 21:29:52.257338 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257285    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 21:29:52.257338 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257322    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 21:29:52.257433 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257328    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 21:29:52.257433 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257362    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 21:29:52.257433 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257370    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 21:29:52.257433 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257371    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 21:29:52.257433 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257401    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 21:29:52.257433 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257402    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 21:29:52.257616 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257434    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 21:29:52.257616 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.257449    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 21:29:52.258831 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.258814    4454 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:52.258948 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.258935    4454 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:52.270797 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.270785    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 21:29:52.270925 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.270914    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 21:29:52.271016 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271001    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qc6g7\""
Nov 03 21:29:52.271082 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271069    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 21:29:52.271143 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271103    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 21:29:52.271192 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271182    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 21:29:52.271322 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271310    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 21:29:52.271431 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271417    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 21:29:52.271550 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271535    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 21:29:52.271627 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271616    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 21:29:52.271682 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271645    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 21:29:52.271682 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271665    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 21:29:52.271761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271696    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qcvh8\""
Nov 03 21:29:52.271761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271703    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 21:29:52.271761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271716    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4tzc5\""
Nov 03 21:29:52.271761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271738    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 21:29:52.271999 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.271986    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bxcrl\""
Nov 03 21:29:52.272101 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.272088    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-fv5g9\""
Nov 03 21:29:52.272144 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.272104    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vzdjb\""
Nov 03 21:29:52.279957 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.279943    4454 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 21:29:52.280015 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:29:52.279986    4454 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-137-212.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:52.280919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.280905    4454 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 21:29:52.280996 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:29:52.280952    4454 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-212.ec2.internal"
Nov 03 21:29:52.286009 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.285980    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:52.290670 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.290656    4454 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 21:29:52.341780 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.341743    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-env-overrides\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.341780 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.341782    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovnkube-script-lib\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.341973 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.341809    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.342049 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342028    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-sys\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.342049 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342031    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-env-overrides\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.342130 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342039    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovnkube-script-lib\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.342130 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342077    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-tuned\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.342130 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342119    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fh5b\" (UniqueName: \"kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b\") pod \"network-check-target-7s62j\" (UID: \"29ad1278-e154-41fb-8648-0adc8d7a648f\") " pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:52.342235 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342139    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/45263f32-066f-4bdf-84b9-a3ebd1f33f90-konnectivity-ca\") pod \"konnectivity-agent-79tpv\" (UID: \"45263f32-066f-4bdf-84b9-a3ebd1f33f90\") " pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:52.342235 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342164    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.342235 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342181    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-socket-dir-parent\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.342235 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342200    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-tuned\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.342380 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342256    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-conf-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.342380 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342314    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-hosts-file\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:52.342380 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342353    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx8fk\" (UniqueName: \"kubernetes.io/projected/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-kube-api-access-dx8fk\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:52.342479 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342403    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.342479 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342422    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/45263f32-066f-4bdf-84b9-a3ebd1f33f90-konnectivity-ca\") pod \"konnectivity-agent-79tpv\" (UID: \"45263f32-066f-4bdf-84b9-a3ebd1f33f90\") " pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:52.342572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342529    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tctvw\" (UniqueName: \"kubernetes.io/projected/15567c09-c50f-4c3b-81b5-5357f08fa543-kube-api-access-tctvw\") pod \"ingress-canary-kbh4v\" (UID: \"15567c09-c50f-4c3b-81b5-5357f08fa543\") " pod="openshift-ingress-canary/ingress-canary-kbh4v"
Nov 03 21:29:52.342604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342589    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.342680 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342667    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gfqr\" (UniqueName: \"kubernetes.io/projected/fe26d35e-47a4-497a-9fe1-64cc807267e9-kube-api-access-2gfqr\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.342727 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342713    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.342772 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342755    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:52.342804 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342780    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-modprobe-d\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.342804 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342794    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-lib-modules\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.342866 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342807    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-cnibin\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.342866 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342832    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-config-volume\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:52.342934 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342876    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovn-node-metrics-cert\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.342967 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342945    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.343000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342972    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-netns\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.343000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342976    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-config-volume\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:52.343000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.342993    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlk69\" (UniqueName: \"kubernetes.io/projected/af4170fc-093a-4066-bf79-4f57388f073d-kube-api-access-jlk69\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:52.343104 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343020    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:52.343104 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343062    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-etc-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.343104 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343083    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysconfig\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.343212 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343106    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.343212 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343107    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-metrics-certs\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:52.343212 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343140    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-tmp-dir\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:52.343317 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343215    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-tmp-dir\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:52.343317 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343243    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qcc8\" (UniqueName: \"kubernetes.io/projected/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-kube-api-access-6qcc8\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:52.343317 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343279    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovn-node-metrics-cert\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.343317 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343289    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.343481 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343370    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysctl-conf\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.343481 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343397    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq4wr\" (UniqueName: \"kubernetes.io/projected/b7e2c320-fbf4-40e6-938d-b4e1377d769c-kube-api-access-dq4wr\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.343481 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343415    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvfs2\" (UniqueName: \"kubernetes.io/projected/8bcb1ef0-aa94-4932-99a6-53178e770a68-kube-api-access-bvfs2\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.343481 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343456    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a287e20-e2d0-451a-97ce-b4fddffeb788-host\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:52.343639 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343489    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-etc-selinux\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.343639 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343532    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-kubernetes\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.343639 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343572    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cnibin\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.343639 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343603    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-systemd\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.343639 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343636    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-var-lib-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.343857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343655    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgc4m\" (UniqueName: \"kubernetes.io/projected/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-api-access-jgc4m\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.343857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343682    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-run\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.343857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343782    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-systemd-units\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.343857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343808    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-cni-bin\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.343857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343838    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-device-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.343857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343856    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-socket-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.344138 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343875    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:52.344138 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343918    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-data-volume\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.344138 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.343961    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/45263f32-066f-4bdf-84b9-a3ebd1f33f90-agent-certs\") pod \"konnectivity-agent-79tpv\" (UID: \"45263f32-066f-4bdf-84b9-a3ebd1f33f90\") " pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:52.344138 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344003    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-data-volume\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.344138 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344041    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-sys-fs\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.344138 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344065    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tfpz\" (UniqueName: \"kubernetes.io/projected/47bb3d82-52e3-4e39-8b1a-1deb886a29db-kube-api-access-7tfpz\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.344138 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344104    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-k8s-cni-cncf-io\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.344138 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344132    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-kubelet\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.344138 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344136    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/45263f32-066f-4bdf-84b9-a3ebd1f33f90-agent-certs\") pod \"konnectivity-agent-79tpv\" (UID: \"45263f32-066f-4bdf-84b9-a3ebd1f33f90\") " pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:29:52.344475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344165    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-hostroot\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.344475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344176    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b978b73d-666a-4c25-b959-74d3e14c7bbf-original-pull-secret\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:52.344475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344185    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af4170fc-093a-4066-bf79-4f57388f073d-host-slash\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:52.344475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344202    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-daemon-config\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.344475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344247    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-multus-certs\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.344475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344305    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15567c09-c50f-4c3b-81b5-5357f08fa543-cert\") pod \"ingress-canary-kbh4v\" (UID: \"15567c09-c50f-4c3b-81b5-5357f08fa543\") " pod="openshift-ingress-canary/ingress-canary-kbh4v"
Nov 03 21:29:52.344475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344359    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lz9n\" (UniqueName: \"kubernetes.io/projected/aa5882bd-1e86-428d-afec-490ebd2da878-kube-api-access-6lz9n\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.344475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344386    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cni-binary-copy\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.344475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344418    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a287e20-e2d0-451a-97ce-b4fddffeb788-serviceca\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:52.344475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344443    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-tmp-dir\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:52.344475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344459    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-daemon-config\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.344475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344466    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-kubelet\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344489    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-var-lib-kubelet\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344527    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cni-binary-copy\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344530    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-system-cni-dir\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344548    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-tmp-dir\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344570    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bcb1ef0-aa94-4932-99a6-53178e770a68-cni-binary-copy\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344583    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15567c09-c50f-4c3b-81b5-5357f08fa543-cert\") pod \"ingress-canary-kbh4v\" (UID: \"15567c09-c50f-4c3b-81b5-5357f08fa543\") " pod="openshift-ingress-canary/ingress-canary-kbh4v"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344598    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-cni-bin\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344664    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-cni-multus\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344702    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8bcb1ef0-aa94-4932-99a6-53178e770a68-cni-binary-copy\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344711    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-slash\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344746    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-node-log\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344771    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344801    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-registration-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344824    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-cni-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344868    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-etc-kubernetes\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344908    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkpsd\" (UniqueName: \"kubernetes.io/projected/3a287e20-e2d0-451a-97ce-b4fddffeb788-kube-api-access-gkpsd\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:52.344941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344935    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-run-netns\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.344957    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-ovn\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345000    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-log-socket\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345023    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysctl-d\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345044    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-host\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345071    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a287e20-e2d0-451a-97ce-b4fddffeb788-serviceca\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345077    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-system-cni-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345103    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b978b73d-666a-4c25-b959-74d3e14c7bbf-kubelet-config\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345117    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345131    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-cni-netd\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345165    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovnkube-config\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345190    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-crio-socket\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345205    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-systemd\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345252    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47bb3d82-52e3-4e39-8b1a-1deb886a29db-tmp\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345292    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af4170fc-093a-4066-bf79-4f57388f073d-iptables-alerter-script\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345321    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82wfl\" (UniqueName: \"kubernetes.io/projected/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-kube-api-access-82wfl\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345346    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b978b73d-666a-4c25-b959-74d3e14c7bbf-dbus\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345372    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345347    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47bb3d82-52e3-4e39-8b1a-1deb886a29db-tmp\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345348    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fe26d35e-47a4-497a-9fe1-64cc807267e9-ovnkube-config\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345517    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af4170fc-093a-4066-bf79-4f57388f073d-iptables-alerter-script\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:52.345578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.345559    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.360419 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.360399    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fh5b\" (UniqueName: \"kubernetes.io/projected/29ad1278-e154-41fb-8648-0adc8d7a648f-kube-api-access-2fh5b\") pod \"network-check-target-7s62j\" (UID: \"29ad1278-e154-41fb-8648-0adc8d7a648f\") " pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:29:52.360739 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.360718    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tctvw\" (UniqueName: \"kubernetes.io/projected/15567c09-c50f-4c3b-81b5-5357f08fa543-kube-api-access-tctvw\") pod \"ingress-canary-kbh4v\" (UID: \"15567c09-c50f-4c3b-81b5-5357f08fa543\") " pod="openshift-ingress-canary/ingress-canary-kbh4v"
Nov 03 21:29:52.360794 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.360718    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlk69\" (UniqueName: \"kubernetes.io/projected/af4170fc-093a-4066-bf79-4f57388f073d-kube-api-access-jlk69\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:52.361400 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.361115    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx8fk\" (UniqueName: \"kubernetes.io/projected/ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a-kube-api-access-dx8fk\") pod \"network-metrics-daemon-4gsg9\" (UID: \"ce0d1d4d-7bb3-42c7-ab87-6c5b460ce65a\") " pod="openshift-multus/network-metrics-daemon-4gsg9"
Nov 03 21:29:52.361400 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.361227    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvfs2\" (UniqueName: \"kubernetes.io/projected/8bcb1ef0-aa94-4932-99a6-53178e770a68-kube-api-access-bvfs2\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.361400 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.361251    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gfqr\" (UniqueName: \"kubernetes.io/projected/fe26d35e-47a4-497a-9fe1-64cc807267e9-kube-api-access-2gfqr\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.361400 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.361350    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tfpz\" (UniqueName: \"kubernetes.io/projected/47bb3d82-52e3-4e39-8b1a-1deb886a29db-kube-api-access-7tfpz\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.361768 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.361743    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq4wr\" (UniqueName: \"kubernetes.io/projected/b7e2c320-fbf4-40e6-938d-b4e1377d769c-kube-api-access-dq4wr\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.361835 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.361778    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82wfl\" (UniqueName: \"kubernetes.io/projected/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-kube-api-access-82wfl\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:52.361878 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.361862    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qcc8\" (UniqueName: \"kubernetes.io/projected/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-kube-api-access-6qcc8\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:52.362049 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.362030    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgc4m\" (UniqueName: \"kubernetes.io/projected/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-kube-api-access-jgc4m\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.362177 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.362163    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkpsd\" (UniqueName: \"kubernetes.io/projected/3a287e20-e2d0-451a-97ce-b4fddffeb788-kube-api-access-gkpsd\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:52.362210 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.362201    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lz9n\" (UniqueName: \"kubernetes.io/projected/aa5882bd-1e86-428d-afec-490ebd2da878-kube-api-access-6lz9n\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.446661 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446583    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-var-lib-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.446661 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446623    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-run\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.446661 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446643    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-systemd-units\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.446661 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446670    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-cni-bin\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446685    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-var-lib-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446703    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-systemd-units\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446720    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-run\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446694    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-device-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446762    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-socket-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446780    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-sys-fs\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446797    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-k8s-cni-cncf-io\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446739    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-cni-bin\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446740    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-device-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446818    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-kubelet\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446854    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-socket-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446854    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-k8s-cni-cncf-io\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446867    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-hostroot\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446886    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-kubelet\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446898    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-hostroot\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446889    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af4170fc-093a-4066-bf79-4f57388f073d-host-slash\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446908    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af4170fc-093a-4066-bf79-4f57388f073d-host-slash\") pod \"iptables-alerter-wk2lw\" (UID: \"af4170fc-093a-4066-bf79-4f57388f073d\") " pod="openshift-network-operator/iptables-alerter-wk2lw"
Nov 03 21:29:52.446933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446932    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-multus-certs\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446950    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-sys-fs\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446961    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-kubelet\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446982    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-multus-certs\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446984    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-var-lib-kubelet\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.446991    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-kubelet\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447010    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-system-cni-dir\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447028    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-cni-bin\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447044    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-var-lib-kubelet\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447077    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-system-cni-dir\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447164    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-cni-bin\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447410    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-cni-multus\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447432    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-slash\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447444    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-var-lib-cni-multus\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447450    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-node-log\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447468    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-node-log\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447486    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447495    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-slash\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447536    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-registration-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.447546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447553    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-cni-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447567    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-etc-kubernetes\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447574    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447581    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-run-netns\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447595    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-ovn\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447614    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-cni-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447624    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-etc-kubernetes\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447636    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-log-socket\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447647    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-registration-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447637    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-run-netns\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447665    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-ovn\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447669    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-log-socket\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447661    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysctl-d\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447711    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-host\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447735    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-system-cni-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447759    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b978b73d-666a-4c25-b959-74d3e14c7bbf-kubelet-config\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447764    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-host\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447767    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysctl-d\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447792    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-system-cni-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447810    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447809    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b978b73d-666a-4c25-b959-74d3e14c7bbf-kubelet-config\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447825    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-cni-netd\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447841    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-crio-socket\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447863    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447855    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-systemd\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447887    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-systemd\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447891    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b978b73d-666a-4c25-b959-74d3e14c7bbf-dbus\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447893    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-cni-netd\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447914    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447925    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-crio-socket\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447931    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-sys\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447937    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b978b73d-666a-4c25-b959-74d3e14c7bbf-dbus\") pod \"global-pull-secret-syncer-vmr9k\" (UID: \"b978b73d-666a-4c25-b959-74d3e14c7bbf\") " pod="kube-system/global-pull-secret-syncer-vmr9k"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447954    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-sys\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447966    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447983    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-socket-dir-parent\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.447998    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-conf-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448015    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-hosts-file\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448044    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448049    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448056    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-socket-dir-parent\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448049    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-multus-conf-dir\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448089    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448089    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0fe02b31-bc0e-4ffc-916d-f6cd60b68a70-hosts-file\") pod \"node-resolver-529hb\" (UID: \"0fe02b31-bc0e-4ffc-916d-f6cd60b68a70\") " pod="openshift-dns/node-resolver-529hb"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448071    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-modprobe-d\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448134    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-lib-modules\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448148    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-cnibin\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448156    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-modprobe-d\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448170    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448183    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-netns\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448197    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448198    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-lib-modules\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448210    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-etc-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448196    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-cnibin\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448231    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysconfig\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448235    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448263    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysctl-conf\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448266    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8bcb1ef0-aa94-4932-99a6-53178e770a68-host-run-netns\") pod \"multus-q2zcj\" (UID: \"8bcb1ef0-aa94-4932-99a6-53178e770a68\") " pod="openshift-multus/multus-q2zcj"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448273    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-etc-openvswitch\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448276    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysconfig\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448293    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a287e20-e2d0-451a-97ce-b4fddffeb788-host\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448294    4454 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448317    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-etc-selinux\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448351    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa5882bd-1e86-428d-afec-490ebd2da878-etc-selinux\") pod \"aws-ebs-csi-driver-node-tl4xk\" (UID: \"aa5882bd-1e86-428d-afec-490ebd2da878\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl4xk"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448355    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a287e20-e2d0-451a-97ce-b4fddffeb788-host\") pod \"node-ca-wjgz5\" (UID: \"3a287e20-e2d0-451a-97ce-b4fddffeb788\") " pod="openshift-image-registry/node-ca-wjgz5"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448381    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-kubernetes\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448390    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-sysctl-conf\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.448363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448417    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cnibin\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.450283 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448440    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-systemd\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.450283 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448460    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7e2c320-fbf4-40e6-938d-b4e1377d769c-cnibin\") pod \"multus-additional-cni-plugins-swgh5\" (UID: \"b7e2c320-fbf4-40e6-938d-b4e1377d769c\") " pod="openshift-multus/multus-additional-cni-plugins-swgh5"
Nov 03 21:29:52.450283 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448441    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47bb3d82-52e3-4e39-8b1a-1deb886a29db-etc-kubernetes\") pod \"tuned-x55l2\" (UID: \"47bb3d82-52e3-4e39-8b1a-1deb886a29db\") " pod="openshift-cluster-node-tuning-operator/tuned-x55l2"
Nov 03 21:29:52.450283 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.448500    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fe26d35e-47a4-497a-9fe1-64cc807267e9-run-systemd\") pod \"ovnkube-node-6xdtq\" (UID: \"fe26d35e-47a4-497a-9fe1-64cc807267e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:52.450474 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.450348    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d09b08fd-64c6-4b9e-ba06-273cbedbd58a-metrics-tls\") pod \"dns-default-sz7tz\" (UID: \"d09b08fd-64c6-4b9e-ba06-273cbedbd58a\") " pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:52.450474 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.450421    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff9199eb-c6a1-4a45-9b7b-862b6e778f4b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-prbdm\" (UID: \"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b\") " pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.533827 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.532961    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:52.533827 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.533343    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-prbdm"
Nov 03 21:29:52.575051 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.575027    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:52.773859 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.773783    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:52.863578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.863551    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sz7tz"]
Nov 03 21:29:52.868386 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:52.868357    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd09b08fd_64c6_4b9e_ba06_273cbedbd58a.slice/crio-f393f8b6ed8cc34be60c74938de477754c1202ef2d0b09da54728c18bcb3b0e4 WatchSource:0}: Error finding container f393f8b6ed8cc34be60c74938de477754c1202ef2d0b09da54728c18bcb3b0e4: Status 404 returned error can't find the container with id f393f8b6ed8cc34be60c74938de477754c1202ef2d0b09da54728c18bcb3b0e4
Nov 03 21:29:52.870609 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.870589    4454 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:29:52.907854 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.907825    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-prbdm"]
Nov 03 21:29:52.912316 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:29:52.912298    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9199eb_c6a1_4a45_9b7b_862b6e778f4b.slice/crio-b9bf4ee33a9fc6366d2c9a1d767dcd66818684ded4513eaf61bc7adf86f09955 WatchSource:0}: Error finding container b9bf4ee33a9fc6366d2c9a1d767dcd66818684ded4513eaf61bc7adf86f09955: Status 404 returned error can't find the container with id b9bf4ee33a9fc6366d2c9a1d767dcd66818684ded4513eaf61bc7adf86f09955
Nov 03 21:29:52.973928 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:52.973900    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:53.074152 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:53.074078    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:53.175019 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:53.174982    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:53.262342 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:53.262312    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-prbdm" event={"ID":"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b","Type":"ContainerStarted","Data":"bd5e5275b6d6441b8c2e063ee05fcf5d5d4f38deda634315cdcb8cecffaeb41b"}
Nov 03 21:29:53.262755 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:53.262371    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-prbdm" event={"ID":"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b","Type":"ContainerStarted","Data":"b9bf4ee33a9fc6366d2c9a1d767dcd66818684ded4513eaf61bc7adf86f09955"}
Nov 03 21:29:53.263452 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:53.263433    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sz7tz" event={"ID":"d09b08fd-64c6-4b9e-ba06-273cbedbd58a","Type":"ContainerStarted","Data":"f393f8b6ed8cc34be60c74938de477754c1202ef2d0b09da54728c18bcb3b0e4"}
Nov 03 21:29:53.376795 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:53.376757    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:53.469394 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:53.469368    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:54.268408 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:54.268363    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-prbdm" event={"ID":"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b","Type":"ContainerStarted","Data":"be8487599d33d3011da20ebbca81e5e2fe724b95ca8513723dae62147437b39b"}
Nov 03 21:29:55.273250 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:55.273210    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sz7tz" event={"ID":"d09b08fd-64c6-4b9e-ba06-273cbedbd58a","Type":"ContainerStarted","Data":"83a1bcfc0b5521482cbf25b4b1ab4c42b089519ce9bc7ad839e5a2407a0d3e9a"}
Nov 03 21:29:55.273576 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:55.273273    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sz7tz" event={"ID":"d09b08fd-64c6-4b9e-ba06-273cbedbd58a","Type":"ContainerStarted","Data":"9ce9671a7b1e99d663a371ffafd8b19ccaf00f721052cf40bee476340984d038"}
Nov 03 21:29:55.273576 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:55.273369    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:29:56.277833 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:56.277797    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-prbdm" event={"ID":"ff9199eb-c6a1-4a45-9b7b-862b6e778f4b","Type":"ContainerStarted","Data":"4039345e128dd641aaee83d31f539ccd40eb0b8aa200f71e0a7ea0d473c2cd70"}
Nov 03 21:29:56.523594 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:56.523558    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:56.626450 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:56.626417    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:56.910971 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:56.910882    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:57.060708 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:57.060671    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:59.430152 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.430113    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:59.529697 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.529665    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:59.643856 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.643820    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:59.754275 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.754203    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:29:59.754446 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.754424    4454 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 21:29:59.815734 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.815705    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq"]
Nov 03 21:29:59.817467 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.817444    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq"
Nov 03 21:29:59.844044 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.844017    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-sbr8c\""
Nov 03 21:29:59.854853 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.854825    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 03 21:29:59.855125 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.855096    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:59.856176 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.856157    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq"]
Nov 03 21:29:59.891258 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.891223    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8acfaa66-20e7-495e-b38c-727510641a05-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-pqsfq\" (UID: \"8acfaa66-20e7-495e-b38c-727510641a05\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq"
Nov 03 21:29:59.901738 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.901709    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6ccd5f8759-n8x2h"]
Nov 03 21:29:59.903365 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.903347    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:29:59.930577 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.930552    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 03 21:29:59.931346 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.931328    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6ccd5f8759-n8x2h"]
Nov 03 21:29:59.939182 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.939159    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 03 21:29:59.939182 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.939177    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-x8jbx\""
Nov 03 21:29:59.939323 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.939214    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 03 21:29:59.939323 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.939261    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 03 21:29:59.962865 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.962839    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:59.991613 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.991594    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7137f08-58ce-4f62-b53f-b5e5c725f003-registry-tls\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:29:59.991702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.991618    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d7137f08-58ce-4f62-b53f-b5e5c725f003-ca-trust-extracted\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:29:59.991702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.991636    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d7137f08-58ce-4f62-b53f-b5e5c725f003-registry-certificates\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:29:59.991702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.991651    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7137f08-58ce-4f62-b53f-b5e5c725f003-trusted-ca\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:29:59.991816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.991706    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8acfaa66-20e7-495e-b38c-727510641a05-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-pqsfq\" (UID: \"8acfaa66-20e7-495e-b38c-727510641a05\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq"
Nov 03 21:29:59.991816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.991737    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d7137f08-58ce-4f62-b53f-b5e5c725f003-image-registry-private-configuration\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:29:59.991816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.991785    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d7137f08-58ce-4f62-b53f-b5e5c725f003-installation-pull-secrets\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:29:59.991945 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.991842    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpj7k\" (UniqueName: \"kubernetes.io/projected/d7137f08-58ce-4f62-b53f-b5e5c725f003-kube-api-access-dpj7k\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:29:59.991945 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.991928    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7137f08-58ce-4f62-b53f-b5e5c725f003-bound-sa-token\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:29:59.994188 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:29:59.994161    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8acfaa66-20e7-495e-b38c-727510641a05-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-pqsfq\" (UID: \"8acfaa66-20e7-495e-b38c-727510641a05\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq"
Nov 03 21:30:00.092664 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.092594    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7137f08-58ce-4f62-b53f-b5e5c725f003-registry-tls\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.092664 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.092624    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d7137f08-58ce-4f62-b53f-b5e5c725f003-ca-trust-extracted\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.092664 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.092639    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d7137f08-58ce-4f62-b53f-b5e5c725f003-registry-certificates\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.092664 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.092655    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7137f08-58ce-4f62-b53f-b5e5c725f003-trusted-ca\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.092925 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.092805    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d7137f08-58ce-4f62-b53f-b5e5c725f003-image-registry-private-configuration\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.092925 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.092850    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d7137f08-58ce-4f62-b53f-b5e5c725f003-installation-pull-secrets\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.093041 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.093011    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpj7k\" (UniqueName: \"kubernetes.io/projected/d7137f08-58ce-4f62-b53f-b5e5c725f003-kube-api-access-dpj7k\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.093107 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.093057    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7137f08-58ce-4f62-b53f-b5e5c725f003-bound-sa-token\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.093107 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.093079    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d7137f08-58ce-4f62-b53f-b5e5c725f003-ca-trust-extracted\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.093647 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.093624    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7137f08-58ce-4f62-b53f-b5e5c725f003-trusted-ca\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.093738 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.093634    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d7137f08-58ce-4f62-b53f-b5e5c725f003-registry-certificates\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.095141 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.095118    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d7137f08-58ce-4f62-b53f-b5e5c725f003-installation-pull-secrets\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.095483 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.095468    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7137f08-58ce-4f62-b53f-b5e5c725f003-registry-tls\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.095632 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.095610    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d7137f08-58ce-4f62-b53f-b5e5c725f003-image-registry-private-configuration\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.113041 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.113020    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7137f08-58ce-4f62-b53f-b5e5c725f003-bound-sa-token\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.113139 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.113118    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpj7k\" (UniqueName: \"kubernetes.io/projected/d7137f08-58ce-4f62-b53f-b5e5c725f003-kube-api-access-dpj7k\") pod \"image-registry-6ccd5f8759-n8x2h\" (UID: \"d7137f08-58ce-4f62-b53f-b5e5c725f003\") " pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.127243 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.127222    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq"
Nov 03 21:30:00.161461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.161430    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:00.212752 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.212725    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:00.261544 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.261496    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:00.411823 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.411797    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq"]
Nov 03 21:30:00.416845 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:30:00.416820    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8acfaa66_20e7_495e_b38c_727510641a05.slice/crio-4cae0aae1d2979f68648b65fdf197a37c13f13a393fa02a9a467532dbc94637e WatchSource:0}: Error finding container 4cae0aae1d2979f68648b65fdf197a37c13f13a393fa02a9a467532dbc94637e: Status 404 returned error can't find the container with id 4cae0aae1d2979f68648b65fdf197a37c13f13a393fa02a9a467532dbc94637e
Nov 03 21:30:00.499831 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:00.499798    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6ccd5f8759-n8x2h"]
Nov 03 21:30:00.504769 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:30:00.504738    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7137f08_58ce_4f62_b53f_b5e5c725f003.slice/crio-c2718cda05967cb6336932b623c7ab67804b1b9ac76eabf03155e5024394d29d WatchSource:0}: Error finding container c2718cda05967cb6336932b623c7ab67804b1b9ac76eabf03155e5024394d29d: Status 404 returned error can't find the container with id c2718cda05967cb6336932b623c7ab67804b1b9ac76eabf03155e5024394d29d
Nov 03 21:30:01.236549 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:01.236504    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:30:01.305346 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:01.305305    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h" event={"ID":"d7137f08-58ce-4f62-b53f-b5e5c725f003","Type":"ContainerStarted","Data":"7f5967b32e2f94d917a2aac1d91b376ea1cd7424ec3a7fca77b94de35d9896d3"}
Nov 03 21:30:01.305551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:01.305374    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h" event={"ID":"d7137f08-58ce-4f62-b53f-b5e5c725f003","Type":"ContainerStarted","Data":"c2718cda05967cb6336932b623c7ab67804b1b9ac76eabf03155e5024394d29d"}
Nov 03 21:30:01.307264 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:01.307231    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq" event={"ID":"8acfaa66-20e7-495e-b38c-727510641a05","Type":"ContainerStarted","Data":"4cae0aae1d2979f68648b65fdf197a37c13f13a393fa02a9a467532dbc94637e"}
Nov 03 21:30:01.345647 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:01.345596    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h" podStartSLOduration=2.345579871 podStartE2EDuration="2.345579871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:29:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:30:01.344977651 +0000 UTC m=+10.223799936" watchObservedRunningTime="2026-11-03 21:30:01.345579871 +0000 UTC m=+10.224402205"
Nov 03 21:30:01.597650 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:01.597626    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:01.697999 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:01.697967    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:02.261468 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:02.261427    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sz7tz_d09b08fd-64c6-4b9e-ba06-273cbedbd58a/dns/0.log"
Nov 03 21:30:02.311578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:02.311543    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq" event={"ID":"8acfaa66-20e7-495e-b38c-727510641a05","Type":"ContainerStarted","Data":"893c5fbea55b3157153a9284469c1822ba4948b92e5b1678518fed3c25156dd3"}
Nov 03 21:30:02.311578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:02.311569    4454 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 21:30:02.311854 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:02.311835    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq"
Nov 03 21:30:02.316081 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:02.316046    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq"
Nov 03 21:30:02.346486 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:02.346446    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pqsfq" podStartSLOduration=2.353042338 podStartE2EDuration="3.346435556s" totalImagesPullingTime="993.393218ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:29:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:30:02.346337049 +0000 UTC m=+11.225159332" watchObservedRunningTime="2026-11-03 21:30:02.346435556 +0000 UTC m=+11.225257840"
Nov 03 21:30:02.461693 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:02.461662    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sz7tz_d09b08fd-64c6-4b9e-ba06-273cbedbd58a/kube-rbac-proxy/0.log"
Nov 03 21:30:02.660240 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:02.660216    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-529hb_0fe02b31-bc0e-4ffc-916d-f6cd60b68a70/dns-node-resolver/0.log"
Nov 03 21:30:03.080138 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:03.080060    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:03.175068 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:03.175039    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:03.260776 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:03.260748    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6ccd5f8759-n8x2h_d7137f08-58ce-4f62-b53f-b5e5c725f003/registry/0.log"
Nov 03 21:30:04.150297 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:04.150263    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:04.252161 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:04.252128    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:04.260234 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:04.260208    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wjgz5_3a287e20-e2d0-451a-97ce-b4fddffeb788/node-ca/0.log"
Nov 03 21:30:04.860847 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:04.860816    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kbh4v_15567c09-c50f-4c3b-81b5-5357f08fa543/serve-healthcheck-canary/0.log"
Nov 03 21:30:05.084100 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:05.084070    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-79tpv"
Nov 03 21:30:05.279589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:05.279497    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sz7tz"
Nov 03 21:30:05.938306 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:05.938272    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:06.224357 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:06.223026    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:06.454015 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:06.453983    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:07.437626 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.437592    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:07.571032 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.570998    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:07.681176 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.681149    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:07.690451 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.690394    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"]
Nov 03 21:30:07.693102 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.693080    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.706791 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.706767    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 03 21:30:07.718817 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.718791    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 03 21:30:07.718964 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.718940    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 21:30:07.719072 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.718965    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 21:30:07.719072 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.719053    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 03 21:30:07.719185 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.719086    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-654qw\""
Nov 03 21:30:07.720083 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.720063    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"]
Nov 03 21:30:07.731458 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.731437    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"]
Nov 03 21:30:07.733223 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.733202    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.756232 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.756200    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-svqc2"]
Nov 03 21:30:07.758447 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.758422    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"]
Nov 03 21:30:07.758570 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.758535    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.790918 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.790887    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:07.808293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.808276    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 03 21:30:07.808684 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.808666    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 03 21:30:07.808684 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.808678    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 03 21:30:07.808818 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.808679    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8jkmv\""
Nov 03 21:30:07.808818 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.808740    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lp7fr\""
Nov 03 21:30:07.808818 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.808768    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 03 21:30:07.808905 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.808828    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 03 21:30:07.808940 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.808911    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 03 21:30:07.847689 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.847661    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-6r4h2\" (UID: \"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.847800 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.847698    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-sys\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.847800 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.847726    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.847800 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.847776    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-6r4h2\" (UID: \"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.847896 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.847814    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-root\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.847929 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.847921    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7073cc7-24ae-4b80-8f87-a217e0073f77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.847960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.847942    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c7073cc7-24ae-4b80-8f87-a217e0073f77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.847991 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.847962    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlkhf\" (UniqueName: \"kubernetes.io/projected/f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb-kube-api-access-mlkhf\") pod \"openshift-state-metrics-7bd7489957-6r4h2\" (UID: \"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.847991 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.847978    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-accelerators-collector-config\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.848051 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.848005    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-textfile\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.848100 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.848087    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c7073cc7-24ae-4b80-8f87-a217e0073f77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.848133 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.848116    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7073cc7-24ae-4b80-8f87-a217e0073f77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.848166 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.848142    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7073cc7-24ae-4b80-8f87-a217e0073f77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.848215 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.848202    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tslq\" (UniqueName: \"kubernetes.io/projected/c7073cc7-24ae-4b80-8f87-a217e0073f77-kube-api-access-5tslq\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.848248 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.848222    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-tls\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.848248 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.848242    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-wtmp\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.848330 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.848264    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-metrics-client-ca\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.848391 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.848365    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k66pf\" (UniqueName: \"kubernetes.io/projected/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-kube-api-access-k66pf\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.848444 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.848432    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-6r4h2\" (UID: \"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.886749 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.886719    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:07.897179 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.897155    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xdtq"
Nov 03 21:30:07.949473 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949394    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-wtmp\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.949473 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949433    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-metrics-client-ca\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.949686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949480    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k66pf\" (UniqueName: \"kubernetes.io/projected/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-kube-api-access-k66pf\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.949686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949534    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-6r4h2\" (UID: \"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.949686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949559    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-6r4h2\" (UID: \"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.949686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949594    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-wtmp\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.949686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949615    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-sys\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.949686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949664    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-sys\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.949686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949665    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.950001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949693    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-6r4h2\" (UID: \"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.950001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949718    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-root\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.950001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949768    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7073cc7-24ae-4b80-8f87-a217e0073f77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.950001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949792    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c7073cc7-24ae-4b80-8f87-a217e0073f77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.950001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949875    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlkhf\" (UniqueName: \"kubernetes.io/projected/f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb-kube-api-access-mlkhf\") pod \"openshift-state-metrics-7bd7489957-6r4h2\" (UID: \"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.950001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.949907    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-accelerators-collector-config\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.950281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.950043    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-textfile\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.950281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.950100    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c7073cc7-24ae-4b80-8f87-a217e0073f77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.950281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.950125    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7073cc7-24ae-4b80-8f87-a217e0073f77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.950281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.950155    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7073cc7-24ae-4b80-8f87-a217e0073f77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.950281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.950201    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tslq\" (UniqueName: \"kubernetes.io/projected/c7073cc7-24ae-4b80-8f87-a217e0073f77-kube-api-access-5tslq\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.950281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.950227    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-tls\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.950584 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.950323    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-metrics-client-ca\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.950584 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.950474    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-6r4h2\" (UID: \"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.950743 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.950724    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-root\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.950814 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.950752    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7073cc7-24ae-4b80-8f87-a217e0073f77-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.951177 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.951132    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c7073cc7-24ae-4b80-8f87-a217e0073f77-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.951262 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.951160    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c7073cc7-24ae-4b80-8f87-a217e0073f77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.951592 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.951525    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-textfile\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.951756 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.951730    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-accelerators-collector-config\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.952396 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.952377    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.952612 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.952583    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-6r4h2\" (UID: \"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.952841 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.952822    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-node-exporter-tls\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.953386 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.953368    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7073cc7-24ae-4b80-8f87-a217e0073f77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.953578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.953558    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7073cc7-24ae-4b80-8f87-a217e0073f77-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.953815 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.953797    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-6r4h2\" (UID: \"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.967728 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.967703    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlkhf\" (UniqueName: \"kubernetes.io/projected/f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb-kube-api-access-mlkhf\") pod \"openshift-state-metrics-7bd7489957-6r4h2\" (UID: \"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:07.967879 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.967857    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k66pf\" (UniqueName: \"kubernetes.io/projected/5484b61e-4f34-4b64-ae30-3c04a5a2e20c-kube-api-access-k66pf\") pod \"node-exporter-svqc2\" (UID: \"5484b61e-4f34-4b64-ae30-3c04a5a2e20c\") " pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:07.969228 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.969205    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tslq\" (UniqueName: \"kubernetes.io/projected/c7073cc7-24ae-4b80-8f87-a217e0073f77-kube-api-access-5tslq\") pod \"kube-state-metrics-5dc8d7d89b-jhvtg\" (UID: \"c7073cc7-24ae-4b80-8f87-a217e0073f77\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:07.987498 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:07.987468    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:08.004790 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:08.004766    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"
Nov 03 21:30:08.044273 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:08.044243    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"
Nov 03 21:30:08.071048 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:08.070973    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-svqc2"
Nov 03 21:30:08.081786 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:30:08.081760    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5484b61e_4f34_4b64_ae30_3c04a5a2e20c.slice/crio-c91053a70c4bcbf54cf74d42cf7f8fb04d8a216f9c2d55aadce528432990b2a6 WatchSource:0}: Error finding container c91053a70c4bcbf54cf74d42cf7f8fb04d8a216f9c2d55aadce528432990b2a6: Status 404 returned error can't find the container with id c91053a70c4bcbf54cf74d42cf7f8fb04d8a216f9c2d55aadce528432990b2a6
Nov 03 21:30:08.087800 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:08.087661    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:08.304122 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:08.304050    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2"]
Nov 03 21:30:08.308922 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:08.308890    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:08.309564 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:30:08.309537    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b7a4fe_6a0c_4dec_a06b_6755c6d02efb.slice/crio-5c1b46a18840b3dda76d40d728262fd31a0ea99d9c80baec11d4e44d8f17dfb5 WatchSource:0}: Error finding container 5c1b46a18840b3dda76d40d728262fd31a0ea99d9c80baec11d4e44d8f17dfb5: Status 404 returned error can't find the container with id 5c1b46a18840b3dda76d40d728262fd31a0ea99d9c80baec11d4e44d8f17dfb5
Nov 03 21:30:08.332481 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:08.332451    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-svqc2" event={"ID":"5484b61e-4f34-4b64-ae30-3c04a5a2e20c","Type":"ContainerStarted","Data":"c91053a70c4bcbf54cf74d42cf7f8fb04d8a216f9c2d55aadce528432990b2a6"}
Nov 03 21:30:08.333644 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:08.333617    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2" event={"ID":"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb","Type":"ContainerStarted","Data":"5c1b46a18840b3dda76d40d728262fd31a0ea99d9c80baec11d4e44d8f17dfb5"}
Nov 03 21:30:08.340603 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:08.340582    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg"]
Nov 03 21:30:08.345703 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:30:08.345679    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7073cc7_24ae_4b80_8f87_a217e0073f77.slice/crio-613c784dd716f2787a4163b8053caea39f696bf392c0f54cc4c8a4bc27925ed1 WatchSource:0}: Error finding container 613c784dd716f2787a4163b8053caea39f696bf392c0f54cc4c8a4bc27925ed1: Status 404 returned error can't find the container with id 613c784dd716f2787a4163b8053caea39f696bf392c0f54cc4c8a4bc27925ed1
Nov 03 21:30:08.908713 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:08.908674    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:09.061325 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:09.061278    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:09.165004 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:09.164898    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:09.340807 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:09.340764    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2" event={"ID":"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb","Type":"ContainerStarted","Data":"6c8b02d31deab1f7335a860a87ddf18bd160dc075ee86ff879a8d6195e946c31"}
Nov 03 21:30:09.340976 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:09.340866    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2" event={"ID":"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb","Type":"ContainerStarted","Data":"4199c693eb163d243833ac218972804293a66af12c275805d9468b283a2b54f6"}
Nov 03 21:30:09.342581 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:09.342552    4454 generic.go:356] "Generic (PLEG): container finished" podID="5484b61e-4f34-4b64-ae30-3c04a5a2e20c" containerID="b8cbd99582a4d2f34545a878ed0674023e72439fa87645ca1bd023018bc13884" exitCode=0
Nov 03 21:30:09.342711 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:09.342652    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-svqc2" event={"ID":"5484b61e-4f34-4b64-ae30-3c04a5a2e20c","Type":"ContainerDied","Data":"b8cbd99582a4d2f34545a878ed0674023e72439fa87645ca1bd023018bc13884"}
Nov 03 21:30:09.344393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:09.344368    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg" event={"ID":"c7073cc7-24ae-4b80-8f87-a217e0073f77","Type":"ContainerStarted","Data":"613c784dd716f2787a4163b8053caea39f696bf392c0f54cc4c8a4bc27925ed1"}
Nov 03 21:30:10.350460 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:10.350425    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg" event={"ID":"c7073cc7-24ae-4b80-8f87-a217e0073f77","Type":"ContainerStarted","Data":"03e1740db62011b286f9c4d85d3b85bd31ac4cb69043e06f7eec4a9c99474b52"}
Nov 03 21:30:10.350902 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:10.350492    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg" event={"ID":"c7073cc7-24ae-4b80-8f87-a217e0073f77","Type":"ContainerStarted","Data":"e154b0d987a0491cd8cba345ad57a4ffef9281bb663ee5abcababb79bcd146e9"}
Nov 03 21:30:10.350902 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:10.350538    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg" event={"ID":"c7073cc7-24ae-4b80-8f87-a217e0073f77","Type":"ContainerStarted","Data":"50f426bba082df434cd98a4e209afd073f80f5518ee008ec9ff57b722a18bc28"}
Nov 03 21:30:10.352426 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:10.352394    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2" event={"ID":"f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb","Type":"ContainerStarted","Data":"5bbd459cec94c966216789830e7953b75ab9e4afcb28158d0a9cbd583cf96aa7"}
Nov 03 21:30:10.354323 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:10.354303    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-svqc2" event={"ID":"5484b61e-4f34-4b64-ae30-3c04a5a2e20c","Type":"ContainerStarted","Data":"b3ebdd7f95e98531dc5897de7a5425fc1e111010207c4331c30a0b2cceba64bb"}
Nov 03 21:30:10.354405 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:10.354344    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-svqc2" event={"ID":"5484b61e-4f34-4b64-ae30-3c04a5a2e20c","Type":"ContainerStarted","Data":"1b222be7984e55860dc35b9ed0d9cf0687fad6b9cc48b8363f88fc7a7170ed18"}
Nov 03 21:30:10.411906 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:10.411836    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jhvtg" podStartSLOduration=2.299649579 podStartE2EDuration="3.411806623s" totalImagesPullingTime="1.112157044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:30:10.399702716 +0000 UTC m=+19.278524990" watchObservedRunningTime="2026-11-03 21:30:10.411806623 +0000 UTC m=+19.290628907"
Nov 03 21:30:10.438847 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:10.438706    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6r4h2" podStartSLOduration=2.4774256279999998 podStartE2EDuration="3.438693076s" totalImagesPullingTime="961.267448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:30:10.438257043 +0000 UTC m=+19.317079326" watchObservedRunningTime="2026-11-03 21:30:10.438693076 +0000 UTC m=+19.317515360"
Nov 03 21:30:10.475681 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:10.475644    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-svqc2" podStartSLOduration=2.451928132 podStartE2EDuration="3.475631722s" totalImagesPullingTime="945.928245ms" totalInitContainerRuntime="77.775345ms" isStatefulPod=true podCreationTimestamp="2026-11-03 21:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:30:10.475030152 +0000 UTC m=+19.353852437" watchObservedRunningTime="2026-11-03 21:30:10.475631722 +0000 UTC m=+19.354454003"
Nov 03 21:30:10.943517 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:10.943473    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:11.043695 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:11.043659    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:11.245348 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:11.245264    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:11.932257 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:11.932223    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:12.033318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.033289    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:12.525084 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.525037    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:12.559300 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.559268    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"]
Nov 03 21:30:12.561168 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.561151    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"
Nov 03 21:30:12.574802 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.574784    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 03 21:30:12.586330 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.586311    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-tvdq4\""
Nov 03 21:30:12.587169 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.587148    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"]
Nov 03 21:30:12.619701 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.619673    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:12.685867 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.685835    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbpnw\" (UniqueName: \"kubernetes.io/projected/d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8-kube-api-access-nbpnw\") pod \"monitoring-plugin-77f4f76594-jt8bk\" (UID: \"d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"
Nov 03 21:30:12.685867 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.685873    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-jt8bk\" (UID: \"d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"
Nov 03 21:30:12.704185 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.704154    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:12.787278 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.787210    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbpnw\" (UniqueName: \"kubernetes.io/projected/d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8-kube-api-access-nbpnw\") pod \"monitoring-plugin-77f4f76594-jt8bk\" (UID: \"d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"
Nov 03 21:30:12.787278 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.787245    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-jt8bk\" (UID: \"d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"
Nov 03 21:30:12.789843 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.789815    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-jt8bk\" (UID: \"d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"
Nov 03 21:30:12.804936 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.804910    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbpnw\" (UniqueName: \"kubernetes.io/projected/d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8-kube-api-access-nbpnw\") pod \"monitoring-plugin-77f4f76594-jt8bk\" (UID: \"d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"
Nov 03 21:30:12.870886 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:12.870859    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"
Nov 03 21:30:13.158842 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:13.158810    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"]
Nov 03 21:30:13.163793 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:30:13.163764    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8902cc5_b1dd_45e0_bdfb_79d23a38b9c8.slice/crio-0bfc283081e53de9b6d4578eb74ae54324b716e1e1d3e0386ea92b6846194a23 WatchSource:0}: Error finding container 0bfc283081e53de9b6d4578eb74ae54324b716e1e1d3e0386ea92b6846194a23: Status 404 returned error can't find the container with id 0bfc283081e53de9b6d4578eb74ae54324b716e1e1d3e0386ea92b6846194a23
Nov 03 21:30:13.365798 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:13.365764    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk" event={"ID":"d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8","Type":"ContainerStarted","Data":"0bfc283081e53de9b6d4578eb74ae54324b716e1e1d3e0386ea92b6846194a23"}
Nov 03 21:30:14.706879 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:14.706841    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:15.375297 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:15.375264    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk" event={"ID":"d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8","Type":"ContainerStarted","Data":"eb60264f11aa5f70b3b15605e4016b93ef94a79365773b08d6b52e66b8eecd05"}
Nov 03 21:30:15.375492 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:15.375479    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"
Nov 03 21:30:15.380064 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:15.380043    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk"
Nov 03 21:30:15.411041 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:15.410996    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jt8bk" podStartSLOduration=2.062276462 podStartE2EDuration="3.410983682s" totalImagesPullingTime="1.34870722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:30:15.410133847 +0000 UTC m=+24.288956150" watchObservedRunningTime="2026-11-03 21:30:15.410983682 +0000 UTC m=+24.289806189"
Nov 03 21:30:16.458879 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:16.458853    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6ccd5f8759-n8x2h"
Nov 03 21:30:16.685626 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:16.685587    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:16.786360 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:16.786282    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:16.885137 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:16.885104    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:19.150706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:19.150668    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:19.251428 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:19.251396    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:20.611428 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:20.611393    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:20.711936 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:20.711907    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:21.240112 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:21.240083    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7s62j"
Nov 03 21:30:21.832942 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:21.832910    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:21.935575 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:21.935529    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:22.374660 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:30:22.374605    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:30:23.468107 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:23.468077    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:23.571467 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:23.571430    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:26.140651 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:26.140612    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:26.254523 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:26.254487    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:26.357874 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:26.357833    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:33.137311 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:33.137278    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:33.239926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:33.239883    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:34.162423 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:34.162333    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:34.254478 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:34.254447    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:34.542803 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:34.542718    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:34.659486 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:34.659452    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:36.401540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:36.401495    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:47.974219 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:47.974175    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:48.402853 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:48.402822    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:52.370646 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:30:52.370607    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:30:53.245877 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.245850    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-567f78586f-6cjcs"]
Nov 03 21:30:53.252630 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.252607    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.267428 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.267401    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 03 21:30:53.267785 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.267401    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 03 21:30:53.279520 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.279472    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 03 21:30:53.279647 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.279575    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-xdpdb\""
Nov 03 21:30:53.279647 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.279591    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 03 21:30:53.280460 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.280438    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-567f78586f-6cjcs"]
Nov 03 21:30:53.291519 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.291478    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:53.347177 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.347147    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 03 21:30:53.359848 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.359827    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 03 21:30:53.369031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.369008    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvf22\" (UniqueName: \"kubernetes.io/projected/5e865acd-e59e-44f5-bf0b-5ea07a883a64-kube-api-access-cvf22\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.369104 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.369037    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e865acd-e59e-44f5-bf0b-5ea07a883a64-serving-certs-ca-bundle\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.369104 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.369056    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e865acd-e59e-44f5-bf0b-5ea07a883a64-telemeter-trusted-ca-bundle\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.369104 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.369073    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5e865acd-e59e-44f5-bf0b-5ea07a883a64-telemeter-client-tls\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.369196 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.369162    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5e865acd-e59e-44f5-bf0b-5ea07a883a64-secret-telemeter-client\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.369230 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.369191    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e865acd-e59e-44f5-bf0b-5ea07a883a64-metrics-client-ca\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.369261 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.369229    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e865acd-e59e-44f5-bf0b-5ea07a883a64-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.369261 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.369253    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5e865acd-e59e-44f5-bf0b-5ea07a883a64-federate-client-tls\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.470453 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.470423    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e865acd-e59e-44f5-bf0b-5ea07a883a64-serving-certs-ca-bundle\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.470882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.470457    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e865acd-e59e-44f5-bf0b-5ea07a883a64-telemeter-trusted-ca-bundle\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.470882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.470485    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5e865acd-e59e-44f5-bf0b-5ea07a883a64-telemeter-client-tls\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.470882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.470531    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5e865acd-e59e-44f5-bf0b-5ea07a883a64-secret-telemeter-client\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.470882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.470557    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e865acd-e59e-44f5-bf0b-5ea07a883a64-metrics-client-ca\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.470882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.470719    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e865acd-e59e-44f5-bf0b-5ea07a883a64-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.470882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.470771    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5e865acd-e59e-44f5-bf0b-5ea07a883a64-federate-client-tls\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.470882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.470837    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvf22\" (UniqueName: \"kubernetes.io/projected/5e865acd-e59e-44f5-bf0b-5ea07a883a64-kube-api-access-cvf22\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.471262 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.471238    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e865acd-e59e-44f5-bf0b-5ea07a883a64-serving-certs-ca-bundle\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.471570 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.471552    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e865acd-e59e-44f5-bf0b-5ea07a883a64-metrics-client-ca\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.471813 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.471789    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e865acd-e59e-44f5-bf0b-5ea07a883a64-telemeter-trusted-ca-bundle\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.473101 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.473072    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5e865acd-e59e-44f5-bf0b-5ea07a883a64-secret-telemeter-client\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.473205 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.473125    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5e865acd-e59e-44f5-bf0b-5ea07a883a64-telemeter-client-tls\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.473205 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.473127    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5e865acd-e59e-44f5-bf0b-5ea07a883a64-federate-client-tls\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.473384 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.473365    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e865acd-e59e-44f5-bf0b-5ea07a883a64-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.488335 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.488316    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvf22\" (UniqueName: \"kubernetes.io/projected/5e865acd-e59e-44f5-bf0b-5ea07a883a64-kube-api-access-cvf22\") pod \"telemeter-client-567f78586f-6cjcs\" (UID: \"5e865acd-e59e-44f5-bf0b-5ea07a883a64\") " pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.567824 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.567749    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs"
Nov 03 21:30:53.886955 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:53.886922    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-567f78586f-6cjcs"]
Nov 03 21:30:53.891702 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:30:53.891674    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e865acd_e59e_44f5_bf0b_5ea07a883a64.slice/crio-be9256cb8aba8e2d3cb5f912bce62db86fb3992d57728cc9cba5796643e3a44b WatchSource:0}: Error finding container be9256cb8aba8e2d3cb5f912bce62db86fb3992d57728cc9cba5796643e3a44b: Status 404 returned error can't find the container with id be9256cb8aba8e2d3cb5f912bce62db86fb3992d57728cc9cba5796643e3a44b
Nov 03 21:30:54.510841 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:54.510791    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs" event={"ID":"5e865acd-e59e-44f5-bf0b-5ea07a883a64","Type":"ContainerStarted","Data":"be9256cb8aba8e2d3cb5f912bce62db86fb3992d57728cc9cba5796643e3a44b"}
Nov 03 21:30:56.518267 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:56.518238    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs" event={"ID":"5e865acd-e59e-44f5-bf0b-5ea07a883a64","Type":"ContainerStarted","Data":"aba32dcbbe95fe5cf2bd1d50395764bcbfd2b429386374c27dc423ce8c9a843b"}
Nov 03 21:30:57.523259 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:57.523220    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs" event={"ID":"5e865acd-e59e-44f5-bf0b-5ea07a883a64","Type":"ContainerStarted","Data":"bfe4bf1d6ad16c8ecb64b25ee7dc982ab2eb3291e2fe683c0a34c17abfa2d1a4"}
Nov 03 21:30:57.523639 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:57.523284    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs" event={"ID":"5e865acd-e59e-44f5-bf0b-5ea07a883a64","Type":"ContainerStarted","Data":"aeea0a1876249ddac1d44fd78f52455c1eb95cc9b9523894639b45ef45a05700"}
Nov 03 21:30:57.588074 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:57.588023    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-567f78586f-6cjcs" podStartSLOduration=2.051098568 podStartE2EDuration="4.588009023s" totalImagesPullingTime="2.536910455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:30:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:30:57.575566073 +0000 UTC m=+66.454388376" watchObservedRunningTime="2026-11-03 21:30:57.588009023 +0000 UTC m=+66.466831340"
Nov 03 21:30:57.877541 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:57.877500    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:57.983696 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:57.983663    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:58.166432 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:58.166347    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:58.248908 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:30:58.248877    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:31:04.148595 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:31:04.148563    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:31:04.237939 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:31:04.237910    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:31:22.371772 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:31:22.371730    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:31:34.675145 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:31:34.675111    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:31:52.368895 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:31:52.368849    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:31:53.819179 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:31:53.819135    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:31:59.633066 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:31:59.633028    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:31:59.739894 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:31:59.739859    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:01.298424 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:32:01.298387    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:01.407164 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:32:01.407125    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:06.399671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:32:06.399590    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:06.498801 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:32:06.498766    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:19.160627 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:32:19.160582    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:19.263802 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:32:19.263765    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:22.369756 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:32:22.369703    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:32:34.802284 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:32:34.802246    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:44.425482 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:32:44.425444    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:44.526256 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:32:44.526220    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:44.611564 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:32:44.611530    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:52.369680 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:32:52.369635    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:33:22.371040 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:33:22.370999    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:33:34.923942 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:33:34.923854    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:38.852865 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:33:38.852824    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:48.535239 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:33:48.535198    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:52.370700 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:33:52.370658    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:33:56.099264 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:33:56.099225    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:56.184484 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:33:56.184450    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:22.369464 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:34:22.369419    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:34:35.034190 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:34:35.034141    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:41.762222 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:34:41.762186    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:41.849859 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:34:41.849827    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:51.226591 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:34:51.226564    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:34:51.227965 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:34:51.227939    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:34:52.369234 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:34:52.369195    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:34:55.891664 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:34:55.891625    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:55.993993 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:34:55.993963    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:03.150249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:03.150211    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:08.891810 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:08.891723    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:09.032561 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.032504    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:09.158589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.158494    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:09.745520 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.745472    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"]
Nov 03 21:35:09.748740 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.748718    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:09.763336 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.763316    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 03 21:35:09.775203 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.775179    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-7w2nj\""
Nov 03 21:35:09.775308 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.775198    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 03 21:35:09.780021 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.780000    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"]
Nov 03 21:35:09.836136 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.836107    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gddbt\" (UniqueName: \"kubernetes.io/projected/f9931d1b-e7e7-4520-b5a3-d7f6860befff-kube-api-access-gddbt\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7\" (UID: \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:09.836290 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.836183    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7\" (UID: \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:09.836290 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.836245    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7\" (UID: \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:09.937394 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.937358    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7\" (UID: \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:09.937394 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.937393    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7\" (UID: \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:09.937814 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.937420    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gddbt\" (UniqueName: \"kubernetes.io/projected/f9931d1b-e7e7-4520-b5a3-d7f6860befff-kube-api-access-gddbt\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7\" (UID: \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:09.937814 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.937794    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7\" (UID: \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:09.937882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.937812    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7\" (UID: \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:09.955582 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:09.955562    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gddbt\" (UniqueName: \"kubernetes.io/projected/f9931d1b-e7e7-4520-b5a3-d7f6860befff-kube-api-access-gddbt\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7\" (UID: \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:10.064268 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.064184    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:10.083997 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.083973    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"]
Nov 03 21:35:10.087339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.087320    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.101949 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.101918    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:10.105030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.105004    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 03 21:35:10.105141 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.105046    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 03 21:35:10.105141 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.105051    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 03 21:35:10.105141 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.105099    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xwclq\""
Nov 03 21:35:10.105311 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.105295    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 03 21:35:10.105311 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.105304    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 03 21:35:10.105416 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.105297    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 03 21:35:10.117960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.117927    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"]
Nov 03 21:35:10.139143 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.139116    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.139290 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.139148    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c497e21a-2793-452a-b533-5a268578e5c3-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.139290 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.139239    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhbhs\" (UniqueName: \"kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-kube-api-access-bhbhs\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.139290 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.139276    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.139415 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.139311    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.139415 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.139339    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.139415 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.139361    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.203394 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.203354    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:10.239787 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.239761    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.239787 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.239789    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c497e21a-2793-452a-b533-5a268578e5c3-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.239966 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.239817    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhbhs\" (UniqueName: \"kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-kube-api-access-bhbhs\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.239966 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.239838    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.239966 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.239867    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.239966 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.239904    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.239966 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.239933    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.240524 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.240494    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.242045 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.242022    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c497e21a-2793-452a-b533-5a268578e5c3-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.242188 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.242172    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.242528 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.242497    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.242600 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.242564    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.258393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.258366    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.258658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.258639    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhbhs\" (UniqueName: \"kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-kube-api-access-bhbhs\") pod \"istiod-openshift-gateway-7876bfcb74-ls4rk\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.358475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.358448    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"]
Nov 03 21:35:10.362987 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:35:10.362963    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9931d1b_e7e7_4520_b5a3_d7f6860befff.slice/crio-074ae06ceb8ddc2a2247ad6972bd8196a077437b561ec30675568c2c61184717 WatchSource:0}: Error finding container 074ae06ceb8ddc2a2247ad6972bd8196a077437b561ec30675568c2c61184717: Status 404 returned error can't find the container with id 074ae06ceb8ddc2a2247ad6972bd8196a077437b561ec30675568c2c61184717
Nov 03 21:35:10.364715 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.364697    4454 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:35:10.400183 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.400161    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:10.703300 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:10.703225    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"]
Nov 03 21:35:10.707801 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:35:10.707774    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc497e21a_2793_452a_b533_5a268578e5c3.slice/crio-db7a09c859861d446f10192d041bc9e8757247103c203d1a9399fe05d532fba1 WatchSource:0}: Error finding container db7a09c859861d446f10192d041bc9e8757247103c203d1a9399fe05d532fba1: Status 404 returned error can't find the container with id db7a09c859861d446f10192d041bc9e8757247103c203d1a9399fe05d532fba1
Nov 03 21:35:11.062738 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:11.062653    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:11.331597 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:11.331520    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk" event={"ID":"c497e21a-2793-452a-b533-5a268578e5c3","Type":"ContainerStarted","Data":"db7a09c859861d446f10192d041bc9e8757247103c203d1a9399fe05d532fba1"}
Nov 03 21:35:11.332639 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:11.332615    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7" event={"ID":"f9931d1b-e7e7-4520-b5a3-d7f6860befff","Type":"ContainerStarted","Data":"074ae06ceb8ddc2a2247ad6972bd8196a077437b561ec30675568c2c61184717"}
Nov 03 21:35:13.614960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:13.614928    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:35:13.615245 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:13.615001    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:35:14.354940 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:14.354904    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk" event={"ID":"c497e21a-2793-452a-b533-5a268578e5c3","Type":"ContainerStarted","Data":"309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276"}
Nov 03 21:35:14.355992 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:14.355929    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:14.358089 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:14.358060    4454 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-ls4rk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:35:14.358303 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:14.358123    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk" podUID="c497e21a-2793-452a-b533-5a268578e5c3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:35:14.401839 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:14.401806    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:14.551402 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:14.551324    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:15.270996 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:15.270964    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:15.359930 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:15.359901    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:35:15.402496 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:15.402464    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:15.424495 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:15.424447    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk" podStartSLOduration=2.520077833 podStartE2EDuration="5.424433177s" totalImagesPullingTime="2.904355344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:35:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:35:14.410209186 +0000 UTC m=+323.289031473" watchObservedRunningTime="2026-11-03 21:35:15.424433177 +0000 UTC m=+324.303255462"
Nov 03 21:35:15.540389 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:15.540298    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:17.367748 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:17.367719    4454 generic.go:356] "Generic (PLEG): container finished" podID="f9931d1b-e7e7-4520-b5a3-d7f6860befff" containerID="2656120b555cbbe01c8ccbb9cde00e0c75910b33112a127ef79c35de7b6a1f61" exitCode=0
Nov 03 21:35:17.368195 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:17.367788    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7" event={"ID":"f9931d1b-e7e7-4520-b5a3-d7f6860befff","Type":"ContainerDied","Data":"2656120b555cbbe01c8ccbb9cde00e0c75910b33112a127ef79c35de7b6a1f61"}
Nov 03 21:35:18.895775 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:18.895735    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:20.379191 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:20.379153    4454 generic.go:356] "Generic (PLEG): container finished" podID="f9931d1b-e7e7-4520-b5a3-d7f6860befff" containerID="a2802d82e0053614dd2ddf8fe239f371702c307aefc3341a51220e3e9d976001" exitCode=0
Nov 03 21:35:20.379613 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:20.379243    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7" event={"ID":"f9931d1b-e7e7-4520-b5a3-d7f6860befff","Type":"ContainerDied","Data":"a2802d82e0053614dd2ddf8fe239f371702c307aefc3341a51220e3e9d976001"}
Nov 03 21:35:22.371544 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:35:22.371483    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:35:24.169816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:24.169783    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:27.408006 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:27.407965    4454 generic.go:356] "Generic (PLEG): container finished" podID="f9931d1b-e7e7-4520-b5a3-d7f6860befff" containerID="6586c331182b7493290a23deaa415d1ce077af15d66a4c4ff04a6026e1310831" exitCode=0
Nov 03 21:35:27.408435 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:27.408019    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7" event={"ID":"f9931d1b-e7e7-4520-b5a3-d7f6860befff","Type":"ContainerDied","Data":"6586c331182b7493290a23deaa415d1ce077af15d66a4c4ff04a6026e1310831"}
Nov 03 21:35:28.532067 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:28.532041    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:28.697989 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:28.697910    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9931d1b-e7e7-4520-b5a3-d7f6860befff-kube-api-access-gddbt\" (UniqueName: \"kubernetes.io/projected/f9931d1b-e7e7-4520-b5a3-d7f6860befff-kube-api-access-gddbt\") pod \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\" (UID: \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\") "
Nov 03 21:35:28.698161 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:28.697992    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-bundle\") pod \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\" (UID: \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\") "
Nov 03 21:35:28.698161 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:28.698018    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-util\" (UniqueName: \"kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-util\") pod \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\" (UID: \"f9931d1b-e7e7-4520-b5a3-d7f6860befff\") "
Nov 03 21:35:28.698610 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:28.698578    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-bundle" pod "f9931d1b-e7e7-4520-b5a3-d7f6860befff" (UID: "f9931d1b-e7e7-4520-b5a3-d7f6860befff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:35:28.700081 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:28.700061    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9931d1b-e7e7-4520-b5a3-d7f6860befff-kube-api-access-gddbt" pod "f9931d1b-e7e7-4520-b5a3-d7f6860befff" (UID: "f9931d1b-e7e7-4520-b5a3-d7f6860befff"). InnerVolumeSpecName "kube-api-access-gddbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:35:28.702636 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:28.702613    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-util" pod "f9931d1b-e7e7-4520-b5a3-d7f6860befff" (UID: "f9931d1b-e7e7-4520-b5a3-d7f6860befff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:35:28.799066 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:28.799034    4454 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-bundle\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:35:28.799066 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:28.799061    4454 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9931d1b-e7e7-4520-b5a3-d7f6860befff-util\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:35:28.799066 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:28.799071    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gddbt\" (UniqueName: \"kubernetes.io/projected/f9931d1b-e7e7-4520-b5a3-d7f6860befff-kube-api-access-gddbt\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:35:29.416283 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:29.416248    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7" event={"ID":"f9931d1b-e7e7-4520-b5a3-d7f6860befff","Type":"ContainerDied","Data":"074ae06ceb8ddc2a2247ad6972bd8196a077437b561ec30675568c2c61184717"}
Nov 03 21:35:29.416449 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:29.416297    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6z9f7"
Nov 03 21:35:29.416449 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:29.416304    4454 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="074ae06ceb8ddc2a2247ad6972bd8196a077437b561ec30675568c2c61184717"
Nov 03 21:35:35.165218 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:35.165179    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:41.694577 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:41.694516    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:45.726139 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:45.726100    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:45.841633 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:45.841601    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:48.393977 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:48.393939    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:48.519823 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:48.519785    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:50.633267 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:50.633227    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:50.797423 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:50.797383    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:52.089605 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:52.089570    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:52.206441 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:52.206390    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:52.373338 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:35:52.373305    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:35:55.876921 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:55.876889    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:57.101249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:57.101218    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:58.095673 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:58.095636    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:58.179110 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:58.179075    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:58.286525 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:58.286469    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:58.843011 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:35:58.842974    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:02.883932 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:02.883891    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:02.986238 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:02.986198    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:04.052021 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.051989    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:04.254304 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.254260    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"]
Nov 03 21:36:04.254557 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.254544    4454 cpu_manager.go:414] "containerMap: removing container" podUID="f9931d1b-e7e7-4520-b5a3-d7f6860befff" containerName="extract"
Nov 03 21:36:04.254604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.254557    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9931d1b-e7e7-4520-b5a3-d7f6860befff" containerName="extract"
Nov 03 21:36:04.254604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.254576    4454 cpu_manager.go:414] "containerMap: removing container" podUID="f9931d1b-e7e7-4520-b5a3-d7f6860befff" containerName="pull"
Nov 03 21:36:04.254604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.254581    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9931d1b-e7e7-4520-b5a3-d7f6860befff" containerName="pull"
Nov 03 21:36:04.254604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.254594    4454 cpu_manager.go:414] "containerMap: removing container" podUID="f9931d1b-e7e7-4520-b5a3-d7f6860befff" containerName="util"
Nov 03 21:36:04.254604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.254599    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9931d1b-e7e7-4520-b5a3-d7f6860befff" containerName="util"
Nov 03 21:36:04.254789 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.254643    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9931d1b-e7e7-4520-b5a3-d7f6860befff" containerName="extract"
Nov 03 21:36:04.256763 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.256747    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.292050 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.292017    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"]
Nov 03 21:36:04.345473 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.345443    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 03 21:36:04.345654 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.345574    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 03 21:36:04.345654 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.345621    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 03 21:36:04.345654 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.345634    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 03 21:36:04.345753 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.345675    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 03 21:36:04.345753 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.345734    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-zrtwp\""
Nov 03 21:36:04.353653 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.353625    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:04.357684 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.357652    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjx45\" (UniqueName: \"kubernetes.io/projected/eb58bbbb-d80f-467c-abcf-268813ebd97c-kube-api-access-xjx45\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.357790 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.357716    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb58bbbb-d80f-467c-abcf-268813ebd97c-metrics-cert\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.357851 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.357787    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eb58bbbb-d80f-467c-abcf-268813ebd97c-manifests\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.357851 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.357836    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb58bbbb-d80f-467c-abcf-268813ebd97c-tmp\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.357971 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.357878    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eb58bbbb-d80f-467c-abcf-268813ebd97c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.357971 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.357910    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb58bbbb-d80f-467c-abcf-268813ebd97c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.458765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.458739    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eb58bbbb-d80f-467c-abcf-268813ebd97c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.458942 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.458769    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb58bbbb-d80f-467c-abcf-268813ebd97c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.458942 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.458800    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjx45\" (UniqueName: \"kubernetes.io/projected/eb58bbbb-d80f-467c-abcf-268813ebd97c-kube-api-access-xjx45\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.458942 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.458830    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb58bbbb-d80f-467c-abcf-268813ebd97c-metrics-cert\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.458942 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.458848    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eb58bbbb-d80f-467c-abcf-268813ebd97c-manifests\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.458942 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.458875    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb58bbbb-d80f-467c-abcf-268813ebd97c-tmp\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.459410 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.459378    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eb58bbbb-d80f-467c-abcf-268813ebd97c-manifests\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.461279 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.461252    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb58bbbb-d80f-467c-abcf-268813ebd97c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.461356 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.461283    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb58bbbb-d80f-467c-abcf-268813ebd97c-metrics-cert\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.461418 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.461400    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb58bbbb-d80f-467c-abcf-268813ebd97c-tmp\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.461769 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.461751    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eb58bbbb-d80f-467c-abcf-268813ebd97c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.471368 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.471343    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:04.480863 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.480839    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjx45\" (UniqueName: \"kubernetes.io/projected/eb58bbbb-d80f-467c-abcf-268813ebd97c-kube-api-access-xjx45\") pod \"opendatahub-operator-controller-manager-5754f75c9f-5xpzp\" (UID: \"eb58bbbb-d80f-467c-abcf-268813ebd97c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.567620 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.567593    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:04.568417 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.568391    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:04.668490 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.668459    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:04.820801 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.820769    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:04.914633 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:04.914556    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"]
Nov 03 21:36:04.917700 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:36:04.917670    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb58bbbb_d80f_467c_abcf_268813ebd97c.slice/crio-8763e4b250c98add5f23af0b301e276e9603a86785dd139277422cc73d59843a WatchSource:0}: Error finding container 8763e4b250c98add5f23af0b301e276e9603a86785dd139277422cc73d59843a: Status 404 returned error can't find the container with id 8763e4b250c98add5f23af0b301e276e9603a86785dd139277422cc73d59843a
Nov 03 21:36:05.135469 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:05.135433    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:05.538996 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:05.538913    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp" event={"ID":"eb58bbbb-d80f-467c-abcf-268813ebd97c","Type":"ContainerStarted","Data":"8763e4b250c98add5f23af0b301e276e9603a86785dd139277422cc73d59843a"}
Nov 03 21:36:05.686462 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:05.686417    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:07.904168 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:07.904085    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:08.278422 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:08.278386    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:08.551751 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:08.551664    4454 generic.go:356] "Generic (PLEG): container finished" podID="eb58bbbb-d80f-467c-abcf-268813ebd97c" containerID="48917975ad5bcf32cbe28fed92ee6b078e5df7ee6d3582610eeb469fdfd6e578" exitCode=0
Nov 03 21:36:08.551893 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:08.551755    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp" event={"ID":"eb58bbbb-d80f-467c-abcf-268813ebd97c","Type":"ContainerDied","Data":"48917975ad5bcf32cbe28fed92ee6b078e5df7ee6d3582610eeb469fdfd6e578"}
Nov 03 21:36:08.893036 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:08.893004    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:09.558759 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:09.558725    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp" event={"ID":"eb58bbbb-d80f-467c-abcf-268813ebd97c","Type":"ContainerStarted","Data":"2d604a6df20326d733c18e593caf9dce5f68ebb336d195e73dfb98e8051cbda2"}
Nov 03 21:36:09.559170 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:09.558951    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:09.635448 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:09.635397    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp" podStartSLOduration=2.100654692 podStartE2EDuration="5.635379525s" totalImagesPullingTime="3.066028816s" totalInitContainerRuntime="468.696017ms" isStatefulPod=false podCreationTimestamp="2026-11-03 21:36:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:36:09.623250991 +0000 UTC m=+378.502073321" watchObservedRunningTime="2026-11-03 21:36:09.635379525 +0000 UTC m=+378.514201808"
Nov 03 21:36:11.103683 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:11.103648    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:11.229076 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:11.229041    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:12.338768 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:12.338719    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:12.492020 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:12.491980    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:13.568051 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:13.568016    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:13.692673 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:13.692637    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:14.324263 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:14.324222    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:14.958944 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:14.958914    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:16.575585 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:16.575546    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:17.529839 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:17.529803    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:20.565154 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:20.565120    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5754f75c9f-5xpzp"
Nov 03 21:36:21.282909 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:21.282869    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:21.373523 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:21.373482    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:21.899007 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:21.898973    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:22.007731 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:22.007694    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:22.369754 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:36:22.369721    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:36:22.543257 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:22.543207    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:22.681614 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:22.681526    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:29.882043 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:29.882007    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:29.981674 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:29.981635    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:31.252730 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.252699    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:31.353329 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.353294    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:31.758994 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.758962    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"]
Nov 03 21:36:31.770184 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.770157    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"
Nov 03 21:36:31.782841 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.782817    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"]
Nov 03 21:36:31.796610 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.795944    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:31.806921 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.806898    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-kck8n\""
Nov 03 21:36:31.806921 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.806915    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 03 21:36:31.807086 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.806932    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 03 21:36:31.879205 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.879171    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjfdl\" (UniqueName: \"kubernetes.io/projected/4cdb1c04-4f27-44a0-b05a-08cb967ccf13-kube-api-access-sjfdl\") pod \"kube-auth-proxy-6b5554b6f8-p8mxx\" (UID: \"4cdb1c04-4f27-44a0-b05a-08cb967ccf13\") " pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"
Nov 03 21:36:31.879205 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.879210    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cdb1c04-4f27-44a0-b05a-08cb967ccf13-tls-certs\") pod \"kube-auth-proxy-6b5554b6f8-p8mxx\" (UID: \"4cdb1c04-4f27-44a0-b05a-08cb967ccf13\") " pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"
Nov 03 21:36:31.879414 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.879282    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4cdb1c04-4f27-44a0-b05a-08cb967ccf13-tmp\") pod \"kube-auth-proxy-6b5554b6f8-p8mxx\" (UID: \"4cdb1c04-4f27-44a0-b05a-08cb967ccf13\") " pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"
Nov 03 21:36:31.904093 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.904055    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:31.979909 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.979879    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4cdb1c04-4f27-44a0-b05a-08cb967ccf13-tmp\") pod \"kube-auth-proxy-6b5554b6f8-p8mxx\" (UID: \"4cdb1c04-4f27-44a0-b05a-08cb967ccf13\") " pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"
Nov 03 21:36:31.980077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.979940    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjfdl\" (UniqueName: \"kubernetes.io/projected/4cdb1c04-4f27-44a0-b05a-08cb967ccf13-kube-api-access-sjfdl\") pod \"kube-auth-proxy-6b5554b6f8-p8mxx\" (UID: \"4cdb1c04-4f27-44a0-b05a-08cb967ccf13\") " pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"
Nov 03 21:36:31.980077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.979969    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cdb1c04-4f27-44a0-b05a-08cb967ccf13-tls-certs\") pod \"kube-auth-proxy-6b5554b6f8-p8mxx\" (UID: \"4cdb1c04-4f27-44a0-b05a-08cb967ccf13\") " pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"
Nov 03 21:36:31.982058 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.982036    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4cdb1c04-4f27-44a0-b05a-08cb967ccf13-tmp\") pod \"kube-auth-proxy-6b5554b6f8-p8mxx\" (UID: \"4cdb1c04-4f27-44a0-b05a-08cb967ccf13\") " pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"
Nov 03 21:36:31.982311 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.982291    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cdb1c04-4f27-44a0-b05a-08cb967ccf13-tls-certs\") pod \"kube-auth-proxy-6b5554b6f8-p8mxx\" (UID: \"4cdb1c04-4f27-44a0-b05a-08cb967ccf13\") " pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"
Nov 03 21:36:31.998740 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:31.998716    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjfdl\" (UniqueName: \"kubernetes.io/projected/4cdb1c04-4f27-44a0-b05a-08cb967ccf13-kube-api-access-sjfdl\") pod \"kube-auth-proxy-6b5554b6f8-p8mxx\" (UID: \"4cdb1c04-4f27-44a0-b05a-08cb967ccf13\") " pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"
Nov 03 21:36:32.072372 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:32.072295    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:32.081185 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:32.081161    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"
Nov 03 21:36:32.238280 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:32.238241    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:32.376947 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:32.376871    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx"]
Nov 03 21:36:32.381499 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:36:32.381472    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cdb1c04_4f27_44a0_b05a_08cb967ccf13.slice/crio-b27e630b1c94ceb2f2cb25b33b3d6650c552c36a4f82709c850f7dc7760465e3 WatchSource:0}: Error finding container b27e630b1c94ceb2f2cb25b33b3d6650c552c36a4f82709c850f7dc7760465e3: Status 404 returned error can't find the container with id b27e630b1c94ceb2f2cb25b33b3d6650c552c36a4f82709c850f7dc7760465e3
Nov 03 21:36:32.643328 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:32.643241    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx" event={"ID":"4cdb1c04-4f27-44a0-b05a-08cb967ccf13","Type":"ContainerStarted","Data":"b27e630b1c94ceb2f2cb25b33b3d6650c552c36a4f82709c850f7dc7760465e3"}
Nov 03 21:36:33.189349 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:33.189315    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:33.356986 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:33.356948    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:33.456673 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:33.456482    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:33.658257 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:33.658218    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:33.917131 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:33.917088    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:34.028575 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.028538    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:34.066675 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.066645    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-gfqbf"]
Nov 03 21:36:34.075656 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.075631    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-gfqbf"
Nov 03 21:36:34.087208 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.087184    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-gfqbf"]
Nov 03 21:36:34.098261 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.097967    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nhc5f\""
Nov 03 21:36:34.126172 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.126142    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 03 21:36:34.126323 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.126225    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 03 21:36:34.182871 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.182696    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:34.198113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.197867    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5psz\" (UniqueName: \"kubernetes.io/projected/a7f98bd1-b4ad-471b-9324-99cf13c65a86-kube-api-access-j5psz\") pod \"authorino-57f7f8659f-gfqbf\" (UID: \"a7f98bd1-b4ad-471b-9324-99cf13c65a86\") " pod="kuadrant-system/authorino-57f7f8659f-gfqbf"
Nov 03 21:36:34.284967 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.284932    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:34.299010 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.298976    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5psz\" (UniqueName: \"kubernetes.io/projected/a7f98bd1-b4ad-471b-9324-99cf13c65a86-kube-api-access-j5psz\") pod \"authorino-57f7f8659f-gfqbf\" (UID: \"a7f98bd1-b4ad-471b-9324-99cf13c65a86\") " pod="kuadrant-system/authorino-57f7f8659f-gfqbf"
Nov 03 21:36:34.318668 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.318642    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5psz\" (UniqueName: \"kubernetes.io/projected/a7f98bd1-b4ad-471b-9324-99cf13c65a86-kube-api-access-j5psz\") pod \"authorino-57f7f8659f-gfqbf\" (UID: \"a7f98bd1-b4ad-471b-9324-99cf13c65a86\") " pod="kuadrant-system/authorino-57f7f8659f-gfqbf"
Nov 03 21:36:34.397269 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.397238    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-gfqbf"
Nov 03 21:36:34.523682 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.523649    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:34.684423 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.684383    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:36:34.689303 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.689281    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"
Nov 03 21:36:34.703886 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.703866    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-gfqbf"]
Nov 03 21:36:34.707491 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.707313    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:36:34.708314 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.708294    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 03 21:36:34.711669 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:36:34.711646    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7f98bd1_b4ad_471b_9324_99cf13c65a86.slice/crio-7f2ef4c64cc430fc4e5a8ae3cf99a06d5c867a92fe992bf8fb2e6ffe3a3773ac WatchSource:0}: Error finding container 7f2ef4c64cc430fc4e5a8ae3cf99a06d5c867a92fe992bf8fb2e6ffe3a3773ac: Status 404 returned error can't find the container with id 7f2ef4c64cc430fc4e5a8ae3cf99a06d5c867a92fe992bf8fb2e6ffe3a3773ac
Nov 03 21:36:34.714186 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.714168    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4sgwz\""
Nov 03 21:36:34.769104 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.769071    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:36:34.803428 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.803393    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgxrf\" (UniqueName: \"kubernetes.io/projected/58409ead-d6be-41bc-abbd-5df50f5ea4bd-kube-api-access-tgxrf\") pod \"limitador-limitador-7c9b87d574-x9hfv\" (UID: \"58409ead-d6be-41bc-abbd-5df50f5ea4bd\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"
Nov 03 21:36:34.803428 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.803431    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/58409ead-d6be-41bc-abbd-5df50f5ea4bd-config-file\") pod \"limitador-limitador-7c9b87d574-x9hfv\" (UID: \"58409ead-d6be-41bc-abbd-5df50f5ea4bd\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"
Nov 03 21:36:34.904667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.904638    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgxrf\" (UniqueName: \"kubernetes.io/projected/58409ead-d6be-41bc-abbd-5df50f5ea4bd-kube-api-access-tgxrf\") pod \"limitador-limitador-7c9b87d574-x9hfv\" (UID: \"58409ead-d6be-41bc-abbd-5df50f5ea4bd\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"
Nov 03 21:36:34.904797 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.904685    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/58409ead-d6be-41bc-abbd-5df50f5ea4bd-config-file\") pod \"limitador-limitador-7c9b87d574-x9hfv\" (UID: \"58409ead-d6be-41bc-abbd-5df50f5ea4bd\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"
Nov 03 21:36:34.905417 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.905394    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/58409ead-d6be-41bc-abbd-5df50f5ea4bd-config-file\") pod \"limitador-limitador-7c9b87d574-x9hfv\" (UID: \"58409ead-d6be-41bc-abbd-5df50f5ea4bd\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"
Nov 03 21:36:34.922665 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:34.922643    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgxrf\" (UniqueName: \"kubernetes.io/projected/58409ead-d6be-41bc-abbd-5df50f5ea4bd-kube-api-access-tgxrf\") pod \"limitador-limitador-7c9b87d574-x9hfv\" (UID: \"58409ead-d6be-41bc-abbd-5df50f5ea4bd\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"
Nov 03 21:36:35.004907 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:35.004828    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"
Nov 03 21:36:35.298125 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:35.298054    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:36:35.302677 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:36:35.302650    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58409ead_d6be_41bc_abbd_5df50f5ea4bd.slice/crio-673d37e6923a8a8524bc09d99a7634b9345ffe2ddc87ec5427b730db314dbcb3 WatchSource:0}: Error finding container 673d37e6923a8a8524bc09d99a7634b9345ffe2ddc87ec5427b730db314dbcb3: Status 404 returned error can't find the container with id 673d37e6923a8a8524bc09d99a7634b9345ffe2ddc87ec5427b730db314dbcb3
Nov 03 21:36:35.335503 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:35.335468    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:35.657952 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:35.657896    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-gfqbf" event={"ID":"a7f98bd1-b4ad-471b-9324-99cf13c65a86","Type":"ContainerStarted","Data":"7f2ef4c64cc430fc4e5a8ae3cf99a06d5c867a92fe992bf8fb2e6ffe3a3773ac"}
Nov 03 21:36:35.659646 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:35.659593    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv" event={"ID":"58409ead-d6be-41bc-abbd-5df50f5ea4bd","Type":"ContainerStarted","Data":"673d37e6923a8a8524bc09d99a7634b9345ffe2ddc87ec5427b730db314dbcb3"}
Nov 03 21:36:35.661585 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:35.661545    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx" event={"ID":"4cdb1c04-4f27-44a0-b05a-08cb967ccf13","Type":"ContainerStarted","Data":"c45440b355485331258d8b82a1b7485ec6ecb0e3180d6f932cb3d2a3aaad45df"}
Nov 03 21:36:35.708206 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:35.708146    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6b5554b6f8-p8mxx" podStartSLOduration=2.238677038 podStartE2EDuration="4.708129599s" totalImagesPullingTime="2.469452561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:36:35.706956332 +0000 UTC m=+404.585778639" watchObservedRunningTime="2026-11-03 21:36:35.708129599 +0000 UTC m=+404.586951883"
Nov 03 21:36:36.352403 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:36.352313    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:36.453732 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:36.453683    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:37.673485 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:37.673441    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv" event={"ID":"58409ead-d6be-41bc-abbd-5df50f5ea4bd","Type":"ContainerStarted","Data":"b824032941b61aa6ca6e31d9af15645f1df32d09e3b3dab4781bc45b3240827b"}
Nov 03 21:36:37.673920 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:37.673557    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"
Nov 03 21:36:37.709837 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:37.709790    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv" podStartSLOduration=1.455366996 podStartE2EDuration="3.709776798s" totalImagesPullingTime="2.254409802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:36:37.709029661 +0000 UTC m=+406.587851959" watchObservedRunningTime="2026-11-03 21:36:37.709776798 +0000 UTC m=+406.588599082"
Nov 03 21:36:37.833420 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:37.833338    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-gfqbf"]
Nov 03 21:36:38.680356 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:38.680321    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-gfqbf" event={"ID":"a7f98bd1-b4ad-471b-9324-99cf13c65a86","Type":"ContainerStarted","Data":"158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b"}
Nov 03 21:36:38.680998 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:38.680503    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-gfqbf" podUID="a7f98bd1-b4ad-471b-9324-99cf13c65a86" containerName="authorino" containerID="cri-o://158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b" gracePeriod=30
Nov 03 21:36:38.715461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:38.715417    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-gfqbf" podStartSLOduration=1.163955068 podStartE2EDuration="4.715402758s" totalImagesPullingTime="3.55144769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:36:38.715183085 +0000 UTC m=+407.594005369" watchObservedRunningTime="2026-11-03 21:36:38.715402758 +0000 UTC m=+407.594225041"
Nov 03 21:36:38.907452 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:38.907427    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-gfqbf"
Nov 03 21:36:39.043592 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:39.043495    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7f98bd1-b4ad-471b-9324-99cf13c65a86-kube-api-access-j5psz\" (UniqueName: \"kubernetes.io/projected/a7f98bd1-b4ad-471b-9324-99cf13c65a86-kube-api-access-j5psz\") pod \"a7f98bd1-b4ad-471b-9324-99cf13c65a86\" (UID: \"a7f98bd1-b4ad-471b-9324-99cf13c65a86\") "
Nov 03 21:36:39.045654 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:39.045626    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f98bd1-b4ad-471b-9324-99cf13c65a86-kube-api-access-j5psz" pod "a7f98bd1-b4ad-471b-9324-99cf13c65a86" (UID: "a7f98bd1-b4ad-471b-9324-99cf13c65a86"). InnerVolumeSpecName "kube-api-access-j5psz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:36:39.144909 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:39.144880    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5psz\" (UniqueName: \"kubernetes.io/projected/a7f98bd1-b4ad-471b-9324-99cf13c65a86-kube-api-access-j5psz\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:36:39.689938 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:39.689892    4454 generic.go:356] "Generic (PLEG): container finished" podID="a7f98bd1-b4ad-471b-9324-99cf13c65a86" containerID="158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b" exitCode=0
Nov 03 21:36:39.690417 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:39.689958    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-gfqbf"
Nov 03 21:36:39.690417 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:39.689980    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-gfqbf" event={"ID":"a7f98bd1-b4ad-471b-9324-99cf13c65a86","Type":"ContainerDied","Data":"158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b"}
Nov 03 21:36:39.690417 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:39.690034    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-gfqbf" event={"ID":"a7f98bd1-b4ad-471b-9324-99cf13c65a86","Type":"ContainerDied","Data":"7f2ef4c64cc430fc4e5a8ae3cf99a06d5c867a92fe992bf8fb2e6ffe3a3773ac"}
Nov 03 21:36:39.690417 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:39.690059    4454 scope.go:122] "RemoveContainer" containerID="158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b"
Nov 03 21:36:39.699383 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:39.699265    4454 scope.go:122] "RemoveContainer" containerID="158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b"
Nov 03 21:36:39.699686 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:36:39.699605    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b\": container with ID starting with 158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b not found: ID does not exist" containerID="158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b"
Nov 03 21:36:39.699686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:39.699643    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b"} err="failed to get container status \"158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b\": rpc error: code = NotFound desc = could not find container \"158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b\": container with ID starting with 158493390b0771023158a2d54ae581f8065afab12c09c73fae01991d0418a92b not found: ID does not exist"
Nov 03 21:36:39.737861 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:39.737834    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-gfqbf"]
Nov 03 21:36:39.739816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:39.739789    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-gfqbf"]
Nov 03 21:36:41.228739 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:41.228702    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7f98bd1-b4ad-471b-9324-99cf13c65a86" path="/var/lib/kubelet/pods/a7f98bd1-b4ad-471b-9324-99cf13c65a86/volumes"
Nov 03 21:36:43.099467 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:43.099431    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:43.188929 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:43.188894    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:43.494815 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:43.494736    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:43.576754 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:43.576715    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:43.903191 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:43.903139    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:44.001461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:44.001424    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:48.681326 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:48.681245    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"
Nov 03 21:36:50.503609 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:50.503577    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:50.602738 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:50.602706    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:52.368955 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:36:52.368917    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:36:53.823766 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:53.823724    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:53.928379 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:53.928349    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:54.634142 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:54.634108    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:55.310894 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:55.310859    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:55.411819 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:55.411791    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:57.752276 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:57.752239    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:57.848752 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:57.848707    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:59.498100 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:59.498067    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:59.637483 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:36:59.637449    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:13.836310 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:13.836274    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:13.936982 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:13.936943    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:14.038293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:14.038263    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:14.124774 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:14.124733    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:17.665330 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:17.665289    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:17.765605 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:17.765563    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:19.118300 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:19.118266    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:19.436698 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:19.436612    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:22.371082 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:37:22.371043    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:37:35.410373 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:35.410338    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:36.695127 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.695092    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"]
Nov 03 21:37:36.695667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.695559    4454 cpu_manager.go:414] "containerMap: removing container" podUID="a7f98bd1-b4ad-471b-9324-99cf13c65a86" containerName="authorino"
Nov 03 21:37:36.695667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.695579    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7f98bd1-b4ad-471b-9324-99cf13c65a86" containerName="authorino"
Nov 03 21:37:36.695667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.695666    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7f98bd1-b4ad-471b-9324-99cf13c65a86" containerName="authorino"
Nov 03 21:37:36.698991 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.698971    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.743526 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.743478    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"]
Nov 03 21:37:36.780683 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.780648    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v85d\" (UniqueName: \"kubernetes.io/projected/0c280d46-c895-4c9d-a21a-2b46f39c59b5-kube-api-access-4v85d\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.780683 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.780688    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0c280d46-c895-4c9d-a21a-2b46f39c59b5-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.780894 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.780711    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c280d46-c895-4c9d-a21a-2b46f39c59b5-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.780894 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.780824    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0c280d46-c895-4c9d-a21a-2b46f39c59b5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.780894 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.780851    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0c280d46-c895-4c9d-a21a-2b46f39c59b5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.780894 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.780878    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c280d46-c895-4c9d-a21a-2b46f39c59b5-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.781025 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.780954    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c280d46-c895-4c9d-a21a-2b46f39c59b5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.882159 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.882124    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v85d\" (UniqueName: \"kubernetes.io/projected/0c280d46-c895-4c9d-a21a-2b46f39c59b5-kube-api-access-4v85d\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.882159 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.882160    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0c280d46-c895-4c9d-a21a-2b46f39c59b5-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.882370 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.882179    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c280d46-c895-4c9d-a21a-2b46f39c59b5-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.882370 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.882240    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0c280d46-c895-4c9d-a21a-2b46f39c59b5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.882370 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.882270    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0c280d46-c895-4c9d-a21a-2b46f39c59b5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.882370 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.882300    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c280d46-c895-4c9d-a21a-2b46f39c59b5-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.882370 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.882331    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c280d46-c895-4c9d-a21a-2b46f39c59b5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.883065 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.883039    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0c280d46-c895-4c9d-a21a-2b46f39c59b5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.884573 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.884553    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0c280d46-c895-4c9d-a21a-2b46f39c59b5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.884673 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.884606    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0c280d46-c895-4c9d-a21a-2b46f39c59b5-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.884803 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.884758    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c280d46-c895-4c9d-a21a-2b46f39c59b5-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.884972 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.884957    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c280d46-c895-4c9d-a21a-2b46f39c59b5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.900396 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.900369    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c280d46-c895-4c9d-a21a-2b46f39c59b5-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:36.900615 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:36.900600    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v85d\" (UniqueName: \"kubernetes.io/projected/0c280d46-c895-4c9d-a21a-2b46f39c59b5-kube-api-access-4v85d\") pod \"istiod-openshift-gateway-7ddd9cfdc8-hr9cx\" (UID: \"0c280d46-c895-4c9d-a21a-2b46f39c59b5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:37.011321 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:37.011244    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:37.315215 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:37.315042    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"]
Nov 03 21:37:37.319814 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:37:37.319789    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c280d46_c895_4c9d_a21a_2b46f39c59b5.slice/crio-59ddac1765d95470f0cee09d1f340c611bd4de48c6159acc70c82f1397732ab7 WatchSource:0}: Error finding container 59ddac1765d95470f0cee09d1f340c611bd4de48c6159acc70c82f1397732ab7: Status 404 returned error can't find the container with id 59ddac1765d95470f0cee09d1f340c611bd4de48c6159acc70c82f1397732ab7
Nov 03 21:37:37.321682 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:37.321654    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:37:37.321766 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:37.321733    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:37:37.868114 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:37.868060    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:37.914573 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:37.914536    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx" event={"ID":"0c280d46-c895-4c9d-a21a-2b46f39c59b5","Type":"ContainerStarted","Data":"9d2ed31d5cf5cfb7f3f7a8ad8d41a70f8d00a73dbdb76d24271bafb0b08c2ae9"}
Nov 03 21:37:37.914732 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:37.914592    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx" event={"ID":"0c280d46-c895-4c9d-a21a-2b46f39c59b5","Type":"ContainerStarted","Data":"59ddac1765d95470f0cee09d1f340c611bd4de48c6159acc70c82f1397732ab7"}
Nov 03 21:37:37.914732 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:37.914689    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:37.955829 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:37.955780    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx" podStartSLOduration=1.955767654 podStartE2EDuration="1.955767654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:37:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:37:37.955224946 +0000 UTC m=+466.834047229" watchObservedRunningTime="2026-11-03 21:37:37.955767654 +0000 UTC m=+466.834589932"
Nov 03 21:37:37.961317 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:37.961284    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:38.054362 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:38.054323    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:38.154026 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:38.153943    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:38.256422 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:38.256389    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:38.369725 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:38.369692    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:38.675338 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:38.675298    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:38.920117 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:38.920093    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hr9cx"
Nov 03 21:37:39.005217 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.005117    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"]
Nov 03 21:37:39.005399 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.005356    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk" podUID="c497e21a-2793-452a-b533-5a268578e5c3" containerName="discovery" containerID="cri-o://309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276" gracePeriod=30
Nov 03 21:37:39.245336 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.245313    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:37:39.306474 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.306408    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-ca-configmap\") pod \"c497e21a-2793-452a-b533-5a268578e5c3\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") "
Nov 03 21:37:39.306474 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.306452    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-cacerts\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-cacerts\") pod \"c497e21a-2793-452a-b533-5a268578e5c3\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") "
Nov 03 21:37:39.306702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.306528    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-istio-token\" (UniqueName: \"kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-istio-token\") pod \"c497e21a-2793-452a-b533-5a268578e5c3\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") "
Nov 03 21:37:39.306702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.306553    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-kube-api-access-bhbhs\" (UniqueName: \"kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-kube-api-access-bhbhs\") pod \"c497e21a-2793-452a-b533-5a268578e5c3\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") "
Nov 03 21:37:39.306702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.306574    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-dns-cert\") pod \"c497e21a-2793-452a-b533-5a268578e5c3\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") "
Nov 03 21:37:39.306702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.306600    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-kubeconfig\") pod \"c497e21a-2793-452a-b533-5a268578e5c3\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") "
Nov 03 21:37:39.306702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.306635    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c497e21a-2793-452a-b533-5a268578e5c3-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c497e21a-2793-452a-b533-5a268578e5c3-local-certs\") pod \"c497e21a-2793-452a-b533-5a268578e5c3\" (UID: \"c497e21a-2793-452a-b533-5a268578e5c3\") "
Nov 03 21:37:39.306953 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.306885    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-ca-configmap" pod "c497e21a-2793-452a-b533-5a268578e5c3" (UID: "c497e21a-2793-452a-b533-5a268578e5c3"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:37:39.308976 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.308916    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-dns-cert" pod "c497e21a-2793-452a-b533-5a268578e5c3" (UID: "c497e21a-2793-452a-b533-5a268578e5c3"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:37:39.309103 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.309014    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-kubeconfig" pod "c497e21a-2793-452a-b533-5a268578e5c3" (UID: "c497e21a-2793-452a-b533-5a268578e5c3"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:37:39.309103 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.309029    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-kube-api-access-bhbhs" pod "c497e21a-2793-452a-b533-5a268578e5c3" (UID: "c497e21a-2793-452a-b533-5a268578e5c3"). InnerVolumeSpecName "kube-api-access-bhbhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:37:39.309103 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.309039    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-istio-token" pod "c497e21a-2793-452a-b533-5a268578e5c3" (UID: "c497e21a-2793-452a-b533-5a268578e5c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:37:39.309258 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.309115    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-cacerts" pod "c497e21a-2793-452a-b533-5a268578e5c3" (UID: "c497e21a-2793-452a-b533-5a268578e5c3"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:37:39.309356 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.309336    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c497e21a-2793-452a-b533-5a268578e5c3-local-certs" pod "c497e21a-2793-452a-b533-5a268578e5c3" (UID: "c497e21a-2793-452a-b533-5a268578e5c3"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:37:39.407398 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.407366    4454 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-cacerts\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:37:39.407398 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.407391    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:37:39.407398 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.407402    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhbhs\" (UniqueName: \"kubernetes.io/projected/c497e21a-2793-452a-b533-5a268578e5c3-kube-api-access-bhbhs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:37:39.407634 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.407410    4454 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-dns-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:37:39.407634 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.407419    4454 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c497e21a-2793-452a-b533-5a268578e5c3-istio-kubeconfig\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:37:39.407634 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.407428    4454 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c497e21a-2793-452a-b533-5a268578e5c3-local-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:37:39.407634 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.407435    4454 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c497e21a-2793-452a-b533-5a268578e5c3-istio-csr-ca-configmap\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:37:39.923593 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.923559    4454 generic.go:356] "Generic (PLEG): container finished" podID="c497e21a-2793-452a-b533-5a268578e5c3" containerID="309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276" exitCode=0
Nov 03 21:37:39.923976 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.923607    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"
Nov 03 21:37:39.923976 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.923639    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk" event={"ID":"c497e21a-2793-452a-b533-5a268578e5c3","Type":"ContainerDied","Data":"309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276"}
Nov 03 21:37:39.923976 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.923687    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk" event={"ID":"c497e21a-2793-452a-b533-5a268578e5c3","Type":"ContainerDied","Data":"db7a09c859861d446f10192d041bc9e8757247103c203d1a9399fe05d532fba1"}
Nov 03 21:37:39.923976 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.923715    4454 scope.go:122] "RemoveContainer" containerID="309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276"
Nov 03 21:37:39.932328 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.932313    4454 scope.go:122] "RemoveContainer" containerID="309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276"
Nov 03 21:37:39.932615 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:37:39.932597    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276\": container with ID starting with 309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276 not found: ID does not exist" containerID="309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276"
Nov 03 21:37:39.932666 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.932623    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276"} err="failed to get container status \"309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276\": rpc error: code = NotFound desc = could not find container \"309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276\": container with ID starting with 309f304a6a6cf88384d5e9c0bb65f664f9abeaf1fc1cbcb93034321352c5a276 not found: ID does not exist"
Nov 03 21:37:39.983373 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.983346    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"]
Nov 03 21:37:39.986823 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:39.986802    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ls4rk"]
Nov 03 21:37:40.510894 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:40.510861    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:40.611027 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:40.610989    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:40.711593 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:40.711559    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:40.812155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:40.812077    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:40.913413 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:40.913380    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:41.013346 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:41.013312    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:41.227206 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:41.227176    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c497e21a-2793-452a-b533-5a268578e5c3" path="/var/lib/kubelet/pods/c497e21a-2793-452a-b533-5a268578e5c3/volumes"
Nov 03 21:37:49.989446 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:49.989403    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:50.094377 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:50.094340    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:51.473310 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.473274    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:51.587039 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.586968    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:51.684492 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.684460    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:51.726392 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.726360    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"]
Nov 03 21:37:51.726699 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.726687    4454 cpu_manager.go:414] "containerMap: removing container" podUID="c497e21a-2793-452a-b533-5a268578e5c3" containerName="discovery"
Nov 03 21:37:51.726747 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.726700    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c497e21a-2793-452a-b533-5a268578e5c3" containerName="discovery"
Nov 03 21:37:51.726789 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.726748    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="c497e21a-2793-452a-b533-5a268578e5c3" containerName="discovery"
Nov 03 21:37:51.731346 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.731325    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:51.745767 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.745746    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:37:51.757382 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.757356    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:37:51.757382 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.757366    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 03 21:37:51.757585 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.757388    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 03 21:37:51.757585 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.757488    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bl9fk\""
Nov 03 21:37:51.758743 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.758722    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"]
Nov 03 21:37:51.804105 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.804076    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:51.810110 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.810089    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8vkb\" (UniqueName: \"kubernetes.io/projected/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-kube-api-access-k8vkb\") pod \"maas-api-6d669c7fcd-l6rj8\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:51.810177 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.810125    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-maas-api-tls\") pod \"maas-api-6d669c7fcd-l6rj8\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:51.810177 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.810149    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-openshift-service-ca\") pod \"maas-api-6d669c7fcd-l6rj8\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:51.810283 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.810269    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-metrics-certs\") pod \"maas-api-6d669c7fcd-l6rj8\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:51.911561 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.911528    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-maas-api-tls\") pod \"maas-api-6d669c7fcd-l6rj8\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:51.911719 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.911573    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-openshift-service-ca\") pod \"maas-api-6d669c7fcd-l6rj8\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:51.911719 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.911594    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-metrics-certs\") pod \"maas-api-6d669c7fcd-l6rj8\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:51.911719 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.911651    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8vkb\" (UniqueName: \"kubernetes.io/projected/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-kube-api-access-k8vkb\") pod \"maas-api-6d669c7fcd-l6rj8\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:51.912298 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.912274    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-openshift-service-ca\") pod \"maas-api-6d669c7fcd-l6rj8\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:51.914042 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.914025    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-maas-api-tls\") pod \"maas-api-6d669c7fcd-l6rj8\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:51.915547 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.915529    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-metrics-certs\") pod \"maas-api-6d669c7fcd-l6rj8\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:51.929855 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:51.929834    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8vkb\" (UniqueName: \"kubernetes.io/projected/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-kube-api-access-k8vkb\") pod \"maas-api-6d669c7fcd-l6rj8\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:52.044222 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:52.044190    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:52.332565 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:52.332446    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"]
Nov 03 21:37:52.336678 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:37:52.336653    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeaf54f0_67ad_4a4a_81e1_1f36e089f6b7.slice/crio-6f52ed00e3d6690745e5a061bc20c8257cd8ca146f803b11f2ee8cba6d46e5e1 WatchSource:0}: Error finding container 6f52ed00e3d6690745e5a061bc20c8257cd8ca146f803b11f2ee8cba6d46e5e1: Status 404 returned error can't find the container with id 6f52ed00e3d6690745e5a061bc20c8257cd8ca146f803b11f2ee8cba6d46e5e1
Nov 03 21:37:52.370017 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:37:52.369984    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:37:52.975231 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:52.975195    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8" event={"ID":"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7","Type":"ContainerStarted","Data":"6f52ed00e3d6690745e5a061bc20c8257cd8ca146f803b11f2ee8cba6d46e5e1"}
Nov 03 21:37:54.454216 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:54.454184    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:37:54.984361 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:54.984312    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8" event={"ID":"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7","Type":"ContainerStarted","Data":"d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6"}
Nov 03 21:37:54.984569 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:54.984440    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:37:55.025293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:37:55.025244    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8" podStartSLOduration=1.87737822 podStartE2EDuration="4.025232424s" totalImagesPullingTime="2.147854204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:37:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:37:55.024145407 +0000 UTC m=+483.902967710" watchObservedRunningTime="2026-11-03 21:37:55.025232424 +0000 UTC m=+483.904054707"
Nov 03 21:38:00.706578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.706542    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"]
Nov 03 21:38:00.710451 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.710429    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.727728 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.727703    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-pgck4\""
Nov 03 21:38:00.741335 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.741308    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"]
Nov 03 21:38:00.884581 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.884540    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.884774 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.884594    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.884774 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.884694    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.884877 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.884794    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np42d\" (UniqueName: \"kubernetes.io/projected/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-kube-api-access-np42d\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.884877 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.884828    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.884877 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.884847    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.885005 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.884889    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.885005 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.884937    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.885065 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.885007    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.985525 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.985430    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.985525 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.985477    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np42d\" (UniqueName: \"kubernetes.io/projected/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-kube-api-access-np42d\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.985525 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.985500    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.985782 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.985531    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.985782 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.985553    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.985782 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.985571    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.985782 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.985608    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.985782 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.985654    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.985782 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.985684    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.986122 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.985987    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.986122 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.986018    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.986122 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.986092    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.986246 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.986207    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.986282 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.986246    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.987920 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.987899    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.988136 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.988117    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:00.993202 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:00.993183    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:38:01.004905 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:01.004886    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-np42d\" (UniqueName: \"kubernetes.io/projected/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-kube-api-access-np42d\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:01.005104 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:01.005087    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e86fb6a8-2f81-4752-92bb-adff8cd0f2cf-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-x7pth\" (UID: \"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:01.023743 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:01.023722    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:01.331087 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:01.330953    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"]
Nov 03 21:38:01.333603 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:38:01.333574    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode86fb6a8_2f81_4752_92bb_adff8cd0f2cf.slice/crio-ab10e68db453a25eb27a4324e29e8461f87a43673c3f3fec2cc232547b2506ad WatchSource:0}: Error finding container ab10e68db453a25eb27a4324e29e8461f87a43673c3f3fec2cc232547b2506ad: Status 404 returned error can't find the container with id ab10e68db453a25eb27a4324e29e8461f87a43673c3f3fec2cc232547b2506ad
Nov 03 21:38:02.017787 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:02.017749    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth" event={"ID":"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf","Type":"ContainerStarted","Data":"ab10e68db453a25eb27a4324e29e8461f87a43673c3f3fec2cc232547b2506ad"}
Nov 03 21:38:03.776297 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:03.776269    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:38:03.776562 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:03.776335    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:38:03.776562 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:03.776364    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:38:04.028188 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:04.028101    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth" event={"ID":"e86fb6a8-2f81-4752-92bb-adff8cd0f2cf","Type":"ContainerStarted","Data":"fa3a869ec9664d7ec921a638d0309931b615d4df0119683085361689c83ce379"}
Nov 03 21:38:04.086683 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:04.086628    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth" podStartSLOduration=1.6467036510000002 podStartE2EDuration="4.08661346s" totalImagesPullingTime="2.439909809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:38:04.085064033 +0000 UTC m=+492.963886318" watchObservedRunningTime="2026-11-03 21:38:04.08661346 +0000 UTC m=+492.965435744"
Nov 03 21:38:05.024592 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:05.024553    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:05.026117 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:05.026092    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x7pth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Nov 03 21:38:05.026231 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:05.026139    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth" podUID="e86fb6a8-2f81-4752-92bb-adff8cd0f2cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Nov 03 21:38:06.024125 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:06.024089    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x7pth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Nov 03 21:38:06.024303 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:06.024140    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth" podUID="e86fb6a8-2f81-4752-92bb-adff8cd0f2cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Nov 03 21:38:07.024290 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:07.024252    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x7pth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Nov 03 21:38:07.024694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:07.024312    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth" podUID="e86fb6a8-2f81-4752-92bb-adff8cd0f2cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Nov 03 21:38:08.024809 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:08.024769    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x7pth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Nov 03 21:38:08.025273 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:08.024829    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth" podUID="e86fb6a8-2f81-4752-92bb-adff8cd0f2cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Nov 03 21:38:09.024057 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:09.024024    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x7pth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Nov 03 21:38:09.024232 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:09.024077    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth" podUID="e86fb6a8-2f81-4752-92bb-adff8cd0f2cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Nov 03 21:38:10.027988 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:10.027955    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:10.028395 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:10.028197    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:10.028904 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:10.028884    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth"
Nov 03 21:38:11.418754 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.418719    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:38:11.520747 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.520708    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:38:11.726182 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.726111    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-bhsjx"]
Nov 03 21:38:11.729458 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.729438    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-bhsjx"
Nov 03 21:38:11.748555 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.748529    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-bhsjx"]
Nov 03 21:38:11.760610 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.760583    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nhc5f\""
Nov 03 21:38:11.779208 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.779180    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvc9j\" (UniqueName: \"kubernetes.io/projected/4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d-kube-api-access-rvc9j\") pod \"authorino-6f6944bcdd-bhsjx\" (UID: \"4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d\") " pod="kuadrant-system/authorino-6f6944bcdd-bhsjx"
Nov 03 21:38:11.880364 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.880333    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvc9j\" (UniqueName: \"kubernetes.io/projected/4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d-kube-api-access-rvc9j\") pod \"authorino-6f6944bcdd-bhsjx\" (UID: \"4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d\") " pod="kuadrant-system/authorino-6f6944bcdd-bhsjx"
Nov 03 21:38:11.904091 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.904063    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvc9j\" (UniqueName: \"kubernetes.io/projected/4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d-kube-api-access-rvc9j\") pod \"authorino-6f6944bcdd-bhsjx\" (UID: \"4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d\") " pod="kuadrant-system/authorino-6f6944bcdd-bhsjx"
Nov 03 21:38:11.919426 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.919400    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-bhsjx"]
Nov 03 21:38:11.919605 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.919591    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-bhsjx"
Nov 03 21:38:11.922785 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.922766    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7458978ff4-76djf"]
Nov 03 21:38:11.926077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.926060    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:11.949949 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.949921    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7458978ff4-76djf"]
Nov 03 21:38:11.981182 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.980977    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0463c19a-5eba-4083-b2cc-1d186b29b59d-openshift-service-ca\") pod \"maas-api-7458978ff4-76djf\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") " pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:11.981182 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.981014    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-maas-api-tls\") pod \"maas-api-7458978ff4-76djf\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") " pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:11.981182 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.981041    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-metrics-certs\") pod \"maas-api-7458978ff4-76djf\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") " pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:11.981182 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:11.981068    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndjxz\" (UniqueName: \"kubernetes.io/projected/0463c19a-5eba-4083-b2cc-1d186b29b59d-kube-api-access-ndjxz\") pod \"maas-api-7458978ff4-76djf\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") " pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:12.082177 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.082143    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0463c19a-5eba-4083-b2cc-1d186b29b59d-openshift-service-ca\") pod \"maas-api-7458978ff4-76djf\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") " pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:12.082177 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.082183    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-maas-api-tls\") pod \"maas-api-7458978ff4-76djf\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") " pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:12.082459 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.082199    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-metrics-certs\") pod \"maas-api-7458978ff4-76djf\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") " pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:12.082459 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.082226    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndjxz\" (UniqueName: \"kubernetes.io/projected/0463c19a-5eba-4083-b2cc-1d186b29b59d-kube-api-access-ndjxz\") pod \"maas-api-7458978ff4-76djf\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") " pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:12.082966 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.082940    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0463c19a-5eba-4083-b2cc-1d186b29b59d-openshift-service-ca\") pod \"maas-api-7458978ff4-76djf\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") " pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:12.084488 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.084468    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-metrics-certs\") pod \"maas-api-7458978ff4-76djf\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") " pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:12.084598 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.084544    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-maas-api-tls\") pod \"maas-api-7458978ff4-76djf\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") " pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:12.105997 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.105971    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndjxz\" (UniqueName: \"kubernetes.io/projected/0463c19a-5eba-4083-b2cc-1d186b29b59d-kube-api-access-ndjxz\") pod \"maas-api-7458978ff4-76djf\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") " pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:12.107003 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.106983    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-86cfdb459f-dlpkg"]
Nov 03 21:38:12.110122 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.110108    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86cfdb459f-dlpkg"
Nov 03 21:38:12.128907 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.128880    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Nov 03 21:38:12.130250 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.130229    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86cfdb459f-dlpkg"]
Nov 03 21:38:12.182871 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.182845    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f46f35a2-d883-4b2e-b198-0a6381704215-tls-cert\") pod \"authorino-86cfdb459f-dlpkg\" (UID: \"f46f35a2-d883-4b2e-b198-0a6381704215\") " pod="kuadrant-system/authorino-86cfdb459f-dlpkg"
Nov 03 21:38:12.183000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.182912    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlrhx\" (UniqueName: \"kubernetes.io/projected/f46f35a2-d883-4b2e-b198-0a6381704215-kube-api-access-zlrhx\") pod \"authorino-86cfdb459f-dlpkg\" (UID: \"f46f35a2-d883-4b2e-b198-0a6381704215\") " pod="kuadrant-system/authorino-86cfdb459f-dlpkg"
Nov 03 21:38:12.214996 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.214960    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-bhsjx"]
Nov 03 21:38:12.217950 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:38:12.217928    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a93ff03_0cc8_43a0_9cfb_afdd348b5a5d.slice/crio-539003e045fb79416a73961d0f7c3c34e4f595404d8bc2904338bc6b68b624d8 WatchSource:0}: Error finding container 539003e045fb79416a73961d0f7c3c34e4f595404d8bc2904338bc6b68b624d8: Status 404 returned error can't find the container with id 539003e045fb79416a73961d0f7c3c34e4f595404d8bc2904338bc6b68b624d8
Nov 03 21:38:12.253324 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.253270    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:12.283575 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.283547    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f46f35a2-d883-4b2e-b198-0a6381704215-tls-cert\") pod \"authorino-86cfdb459f-dlpkg\" (UID: \"f46f35a2-d883-4b2e-b198-0a6381704215\") " pod="kuadrant-system/authorino-86cfdb459f-dlpkg"
Nov 03 21:38:12.283689 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.283630    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlrhx\" (UniqueName: \"kubernetes.io/projected/f46f35a2-d883-4b2e-b198-0a6381704215-kube-api-access-zlrhx\") pod \"authorino-86cfdb459f-dlpkg\" (UID: \"f46f35a2-d883-4b2e-b198-0a6381704215\") " pod="kuadrant-system/authorino-86cfdb459f-dlpkg"
Nov 03 21:38:12.285811 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.285790    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f46f35a2-d883-4b2e-b198-0a6381704215-tls-cert\") pod \"authorino-86cfdb459f-dlpkg\" (UID: \"f46f35a2-d883-4b2e-b198-0a6381704215\") " pod="kuadrant-system/authorino-86cfdb459f-dlpkg"
Nov 03 21:38:12.303309 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.303281    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlrhx\" (UniqueName: \"kubernetes.io/projected/f46f35a2-d883-4b2e-b198-0a6381704215-kube-api-access-zlrhx\") pod \"authorino-86cfdb459f-dlpkg\" (UID: \"f46f35a2-d883-4b2e-b198-0a6381704215\") " pod="kuadrant-system/authorino-86cfdb459f-dlpkg"
Nov 03 21:38:12.422618 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.422585    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86cfdb459f-dlpkg"
Nov 03 21:38:12.543026 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.543004    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7458978ff4-76djf"]
Nov 03 21:38:12.545210 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:38:12.545184    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0463c19a_5eba_4083_b2cc_1d186b29b59d.slice/crio-3c92df2a4690ae9bcdbbbe9a4966f0f36b0231c7961456f8a786e14f1e68c52f WatchSource:0}: Error finding container 3c92df2a4690ae9bcdbbbe9a4966f0f36b0231c7961456f8a786e14f1e68c52f: Status 404 returned error can't find the container with id 3c92df2a4690ae9bcdbbbe9a4966f0f36b0231c7961456f8a786e14f1e68c52f
Nov 03 21:38:12.716533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:12.713866    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86cfdb459f-dlpkg"]
Nov 03 21:38:12.720011 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:38:12.719980    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf46f35a2_d883_4b2e_b198_0a6381704215.slice/crio-6aa51ae8fdbb25895d0f1b8909fc7d97667969caf224e26010f91c898fe31028 WatchSource:0}: Error finding container 6aa51ae8fdbb25895d0f1b8909fc7d97667969caf224e26010f91c898fe31028: Status 404 returned error can't find the container with id 6aa51ae8fdbb25895d0f1b8909fc7d97667969caf224e26010f91c898fe31028
Nov 03 21:38:13.063586 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:13.062964    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf" event={"ID":"0463c19a-5eba-4083-b2cc-1d186b29b59d","Type":"ContainerStarted","Data":"50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea"}
Nov 03 21:38:13.063586 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:13.063071    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf" event={"ID":"0463c19a-5eba-4083-b2cc-1d186b29b59d","Type":"ContainerStarted","Data":"3c92df2a4690ae9bcdbbbe9a4966f0f36b0231c7961456f8a786e14f1e68c52f"}
Nov 03 21:38:13.063586 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:13.063103    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:13.064812 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:13.064743    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-bhsjx" event={"ID":"4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d","Type":"ContainerStarted","Data":"365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db"}
Nov 03 21:38:13.064812 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:13.064786    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-bhsjx" event={"ID":"4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d","Type":"ContainerStarted","Data":"539003e045fb79416a73961d0f7c3c34e4f595404d8bc2904338bc6b68b624d8"}
Nov 03 21:38:13.064812 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:13.064774    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6f6944bcdd-bhsjx" podUID="4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d" containerName="authorino" containerID="cri-o://365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db" gracePeriod=30
Nov 03 21:38:13.066151 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:13.066128    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86cfdb459f-dlpkg" event={"ID":"f46f35a2-d883-4b2e-b198-0a6381704215","Type":"ContainerStarted","Data":"6aa51ae8fdbb25895d0f1b8909fc7d97667969caf224e26010f91c898fe31028"}
Nov 03 21:38:13.103939 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:13.103891    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf" podStartSLOduration=1.815902034 podStartE2EDuration="2.103877713s" totalImagesPullingTime="287.975679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:38:13.102758433 +0000 UTC m=+501.981580740" watchObservedRunningTime="2026-11-03 21:38:13.103877713 +0000 UTC m=+501.982699998"
Nov 03 21:38:13.324156 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:13.324094    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-bhsjx"
Nov 03 21:38:13.393209 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:13.393174    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d-kube-api-access-rvc9j\" (UniqueName: \"kubernetes.io/projected/4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d-kube-api-access-rvc9j\") pod \"4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d\" (UID: \"4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d\") "
Nov 03 21:38:13.395140 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:13.395119    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d-kube-api-access-rvc9j" pod "4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d" (UID: "4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d"). InnerVolumeSpecName "kube-api-access-rvc9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:38:13.494492 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:13.494456    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvc9j\" (UniqueName: \"kubernetes.io/projected/4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d-kube-api-access-rvc9j\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:38:14.071184 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:14.071151    4454 generic.go:356] "Generic (PLEG): container finished" podID="4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d" containerID="365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db" exitCode=0
Nov 03 21:38:14.071357 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:14.071196    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-bhsjx"
Nov 03 21:38:14.071357 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:14.071241    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-bhsjx" event={"ID":"4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d","Type":"ContainerDied","Data":"365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db"}
Nov 03 21:38:14.071357 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:14.071268    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-bhsjx" event={"ID":"4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d","Type":"ContainerDied","Data":"539003e045fb79416a73961d0f7c3c34e4f595404d8bc2904338bc6b68b624d8"}
Nov 03 21:38:14.071357 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:14.071283    4454 scope.go:122] "RemoveContainer" containerID="365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db"
Nov 03 21:38:14.072918 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:14.072894    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86cfdb459f-dlpkg" event={"ID":"f46f35a2-d883-4b2e-b198-0a6381704215","Type":"ContainerStarted","Data":"127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df"}
Nov 03 21:38:14.079855 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:14.079838    4454 scope.go:122] "RemoveContainer" containerID="365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db"
Nov 03 21:38:14.080102 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:38:14.080084    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db\": container with ID starting with 365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db not found: ID does not exist" containerID="365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db"
Nov 03 21:38:14.080139 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:14.080111    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db"} err="failed to get container status \"365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db\": rpc error: code = NotFound desc = could not find container \"365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db\": container with ID starting with 365cbef0b25ed28597f1d3c71d98d6224170f1f3464a4564ebea59077c84b2db not found: ID does not exist"
Nov 03 21:38:14.124474 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:14.124443    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-bhsjx"]
Nov 03 21:38:14.127569 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:14.127548    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-bhsjx"]
Nov 03 21:38:14.165726 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:14.165633    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-86cfdb459f-dlpkg" podStartSLOduration=1.7809718270000001 podStartE2EDuration="2.165619499s" totalImagesPullingTime="384.647672ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:38:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:38:14.164817328 +0000 UTC m=+503.043639625" watchObservedRunningTime="2026-11-03 21:38:14.165619499 +0000 UTC m=+503.044441783"
Nov 03 21:38:15.227798 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:15.227766    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d" path="/var/lib/kubelet/pods/4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d/volumes"
Nov 03 21:38:19.078424 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.078392    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 21:38:19.146011 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.145977    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"]
Nov 03 21:38:19.146246 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.146220    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8" podUID="deaf54f0-67ad-4a4a-81e1-1f36e089f6b7" containerName="maas-api" containerID="cri-o://d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6" gracePeriod=30
Nov 03 21:38:19.384465 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.384442    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:38:19.449632 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.449603    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-openshift-service-ca\") pod \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") "
Nov 03 21:38:19.449772 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.449648    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-kube-api-access-k8vkb\" (UniqueName: \"kubernetes.io/projected/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-kube-api-access-k8vkb\") pod \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") "
Nov 03 21:38:19.449772 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.449676    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-maas-api-tls\") pod \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") "
Nov 03 21:38:19.449878 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.449855    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-metrics-certs\") pod \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\" (UID: \"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7\") "
Nov 03 21:38:19.450008 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.449984    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-openshift-service-ca" pod "deaf54f0-67ad-4a4a-81e1-1f36e089f6b7" (UID: "deaf54f0-67ad-4a4a-81e1-1f36e089f6b7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:38:19.450214 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.450191    4454 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-openshift-service-ca\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:38:19.451806 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.451782    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-maas-api-tls" pod "deaf54f0-67ad-4a4a-81e1-1f36e089f6b7" (UID: "deaf54f0-67ad-4a4a-81e1-1f36e089f6b7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:38:19.451899 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.451829    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-metrics-certs" pod "deaf54f0-67ad-4a4a-81e1-1f36e089f6b7" (UID: "deaf54f0-67ad-4a4a-81e1-1f36e089f6b7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:38:19.451899 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.451848    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-kube-api-access-k8vkb" pod "deaf54f0-67ad-4a4a-81e1-1f36e089f6b7" (UID: "deaf54f0-67ad-4a4a-81e1-1f36e089f6b7"). InnerVolumeSpecName "kube-api-access-k8vkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:38:19.551221 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.551190    4454 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-metrics-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:38:19.551221 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.551215    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8vkb\" (UniqueName: \"kubernetes.io/projected/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-kube-api-access-k8vkb\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:38:19.551221 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:19.551225    4454 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7-maas-api-tls\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:38:20.098079 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:20.098043    4454 generic.go:356] "Generic (PLEG): container finished" podID="deaf54f0-67ad-4a4a-81e1-1f36e089f6b7" containerID="d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6" exitCode=0
Nov 03 21:38:20.098609 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:20.098088    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8" event={"ID":"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7","Type":"ContainerDied","Data":"d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6"}
Nov 03 21:38:20.098609 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:20.098092    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"
Nov 03 21:38:20.098609 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:20.098129    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8" event={"ID":"deaf54f0-67ad-4a4a-81e1-1f36e089f6b7","Type":"ContainerDied","Data":"6f52ed00e3d6690745e5a061bc20c8257cd8ca146f803b11f2ee8cba6d46e5e1"}
Nov 03 21:38:20.098609 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:20.098151    4454 scope.go:122] "RemoveContainer" containerID="d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6"
Nov 03 21:38:20.106297 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:20.106280    4454 scope.go:122] "RemoveContainer" containerID="d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6"
Nov 03 21:38:20.106551 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:38:20.106531    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6\": container with ID starting with d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6 not found: ID does not exist" containerID="d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6"
Nov 03 21:38:20.106614 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:20.106565    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6"} err="failed to get container status \"d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6\": rpc error: code = NotFound desc = could not find container \"d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6\": container with ID starting with d76ffc97e0cce110af68d6366c6032431819d1c700d984ab08c8ead9c8358ba6 not found: ID does not exist"
Nov 03 21:38:20.150098 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:20.150073    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"]
Nov 03 21:38:20.162892 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:20.162872    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-l6rj8"]
Nov 03 21:38:21.228155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:21.228125    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="deaf54f0-67ad-4a4a-81e1-1f36e089f6b7" path="/var/lib/kubelet/pods/deaf54f0-67ad-4a4a-81e1-1f36e089f6b7/volumes"
Nov 03 21:38:22.369747 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:38:22.369712    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:38:35.525353 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:35.525317    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:38:37.211457 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:37.211428    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:38:45.831411 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.831375    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"]
Nov 03 21:38:45.832087 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.831931    4454 cpu_manager.go:414] "containerMap: removing container" podUID="deaf54f0-67ad-4a4a-81e1-1f36e089f6b7" containerName="maas-api"
Nov 03 21:38:45.832087 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.831951    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="deaf54f0-67ad-4a4a-81e1-1f36e089f6b7" containerName="maas-api"
Nov 03 21:38:45.832087 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.831968    4454 cpu_manager.go:414] "containerMap: removing container" podUID="4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d" containerName="authorino"
Nov 03 21:38:45.832087 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.831976    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d" containerName="authorino"
Nov 03 21:38:45.832087 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.832051    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="deaf54f0-67ad-4a4a-81e1-1f36e089f6b7" containerName="maas-api"
Nov 03 21:38:45.832087 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.832061    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a93ff03-0cc8-43a0-9cfb-afdd348b5a5d" containerName="authorino"
Nov 03 21:38:45.834828 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.834807    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:45.849583 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.849559    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"]
Nov 03 21:38:45.852447 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.852429    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:45.866235 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.866211    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"]
Nov 03 21:38:45.879442 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.879417    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"]
Nov 03 21:38:45.965020 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.964983    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:45.965249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965032    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:45.965249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965048    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:45.965249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965143    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea097b76-14e5-48e9-9966-ed40754d66f7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:45.965249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965204    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:45.965249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965247    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:45.965409 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965324    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea097b76-14e5-48e9-9966-ed40754d66f7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:45.965409 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965347    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6174dd8c-e88b-48eb-8ba6-9935eb711810-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:45.965409 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965367    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea097b76-14e5-48e9-9966-ed40754d66f7-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:45.965409 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965384    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6174dd8c-e88b-48eb-8ba6-9935eb711810-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:45.965552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965476    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:45.965552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965495    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7bts\" (UniqueName: \"kubernetes.io/projected/ea097b76-14e5-48e9-9966-ed40754d66f7-kube-api-access-s7bts\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:45.965552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965537    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:45.965648 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965608    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:45.965648 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965625    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:45.965648 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965639    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6174dd8c-e88b-48eb-8ba6-9935eb711810-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:45.965739 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965653    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwkkf\" (UniqueName: \"kubernetes.io/projected/6174dd8c-e88b-48eb-8ba6-9935eb711810-kube-api-access-qwkkf\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:45.965739 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:45.965670    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.066564 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.066524    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.066564 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.066567    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.066754 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.066583    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.066754 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.066710    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6174dd8c-e88b-48eb-8ba6-9935eb711810-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.066827 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.066762    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwkkf\" (UniqueName: \"kubernetes.io/projected/6174dd8c-e88b-48eb-8ba6-9935eb711810-kube-api-access-qwkkf\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.066827 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.066786    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.066827 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.066818    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067037 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.066870    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.067037 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.066899    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067037 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.066928    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea097b76-14e5-48e9-9966-ed40754d66f7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067037 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.066955    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.067037 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.066958    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067037 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067003    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.067037 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067030    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.067393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067079    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.067393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067086    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea097b76-14e5-48e9-9966-ed40754d66f7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067127    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6174dd8c-e88b-48eb-8ba6-9935eb711810-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.067393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067156    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea097b76-14e5-48e9-9966-ed40754d66f7-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067184    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6174dd8c-e88b-48eb-8ba6-9935eb711810-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.067393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067227    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067255    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7bts\" (UniqueName: \"kubernetes.io/projected/ea097b76-14e5-48e9-9966-ed40754d66f7-kube-api-access-s7bts\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067282    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067296    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067365    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067977 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067451    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067977 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067455    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6174dd8c-e88b-48eb-8ba6-9935eb711810-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.067977 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067789    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea097b76-14e5-48e9-9966-ed40754d66f7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.067977 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.067935    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.069460 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.069440    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6174dd8c-e88b-48eb-8ba6-9935eb711810-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.069632 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.069613    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea097b76-14e5-48e9-9966-ed40754d66f7-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.069898 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.069879    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6174dd8c-e88b-48eb-8ba6-9935eb711810-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.069937 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.069879    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea097b76-14e5-48e9-9966-ed40754d66f7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.085843 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.085788    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea097b76-14e5-48e9-9966-ed40754d66f7-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.086051 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.086032    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwkkf\" (UniqueName: \"kubernetes.io/projected/6174dd8c-e88b-48eb-8ba6-9935eb711810-kube-api-access-qwkkf\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.086660 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.086638    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6174dd8c-e88b-48eb-8ba6-9935eb711810-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-lq9lw\" (UID: \"6174dd8c-e88b-48eb-8ba6-9935eb711810\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.086660 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.086653    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7bts\" (UniqueName: \"kubernetes.io/projected/ea097b76-14e5-48e9-9966-ed40754d66f7-kube-api-access-s7bts\") pod \"maas-default-gateway-openshift-default-75d7c785f-x4f2g\" (UID: \"ea097b76-14e5-48e9-9966-ed40754d66f7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.146744 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.146716    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:46.164074 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.164047    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:46.451767 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.451730    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"]
Nov 03 21:38:46.455621 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:38:46.455594    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6174dd8c_e88b_48eb_8ba6_9935eb711810.slice/crio-ffcf1fe7c1fae040f03f2bfa81c964e558063ad87606dec00521b0b7dcf53389 WatchSource:0}: Error finding container ffcf1fe7c1fae040f03f2bfa81c964e558063ad87606dec00521b0b7dcf53389: Status 404 returned error can't find the container with id ffcf1fe7c1fae040f03f2bfa81c964e558063ad87606dec00521b0b7dcf53389
Nov 03 21:38:46.457780 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.457745    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:38:46.457840 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.457823    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:38:46.457896 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.457882    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:38:46.486167 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.486094    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"]
Nov 03 21:38:46.491463 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:38:46.491430    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea097b76_14e5_48e9_9966_ed40754d66f7.slice/crio-f36a96047d58c1e01d585e1858e80201997816610421f10aa72366debd77e2ff WatchSource:0}: Error finding container f36a96047d58c1e01d585e1858e80201997816610421f10aa72366debd77e2ff: Status 404 returned error can't find the container with id f36a96047d58c1e01d585e1858e80201997816610421f10aa72366debd77e2ff
Nov 03 21:38:46.493650 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.493613    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:38:46.493753 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.493738    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:38:46.493825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:46.493808    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:38:47.215965 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:47.215553    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw" event={"ID":"6174dd8c-e88b-48eb-8ba6-9935eb711810","Type":"ContainerStarted","Data":"de2c2837d53c92b85c71b3e7ac0e386202c0cd7bc18416fe057fababca7ea070"}
Nov 03 21:38:47.215965 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:47.215606    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw" event={"ID":"6174dd8c-e88b-48eb-8ba6-9935eb711810","Type":"ContainerStarted","Data":"ffcf1fe7c1fae040f03f2bfa81c964e558063ad87606dec00521b0b7dcf53389"}
Nov 03 21:38:47.217091 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:47.217068    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g" event={"ID":"ea097b76-14e5-48e9-9966-ed40754d66f7","Type":"ContainerStarted","Data":"2c06d96232de00ca5cfb8fa4872b6f704273a8eb0829b5759a8da6e1eeac042d"}
Nov 03 21:38:47.217153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:47.217110    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g" event={"ID":"ea097b76-14e5-48e9-9966-ed40754d66f7","Type":"ContainerStarted","Data":"f36a96047d58c1e01d585e1858e80201997816610421f10aa72366debd77e2ff"}
Nov 03 21:38:47.327826 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:47.327770    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g" podStartSLOduration=2.327755617 podStartE2EDuration="2.327755617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:38:47.326949938 +0000 UTC m=+536.205772232" watchObservedRunningTime="2026-11-03 21:38:47.327755617 +0000 UTC m=+536.206577901"
Nov 03 21:38:47.328304 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:47.328271    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw" podStartSLOduration=2.328262031 podStartE2EDuration="2.328262031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:38:47.287639482 +0000 UTC m=+536.166461790" watchObservedRunningTime="2026-11-03 21:38:47.328262031 +0000 UTC m=+536.207084316"
Nov 03 21:38:48.147347 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:48.147304    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:48.148955 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:48.148676    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lq9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Nov 03 21:38:48.148955 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:48.148722    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw" podUID="6174dd8c-e88b-48eb-8ba6-9935eb711810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Nov 03 21:38:48.169931 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:48.168897    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:48.170221 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:48.170197    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x4f2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Nov 03 21:38:48.170415 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:48.170388    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g" podUID="ea097b76-14e5-48e9-9966-ed40754d66f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Nov 03 21:38:49.148010 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:49.147964    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lq9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Nov 03 21:38:49.148464 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:49.148030    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw" podUID="6174dd8c-e88b-48eb-8ba6-9935eb711810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Nov 03 21:38:49.164900 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:49.164864    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x4f2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Nov 03 21:38:49.165109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:49.165081    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g" podUID="ea097b76-14e5-48e9-9966-ed40754d66f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Nov 03 21:38:50.147440 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:50.147400    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lq9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Nov 03 21:38:50.147649 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:50.147477    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw" podUID="6174dd8c-e88b-48eb-8ba6-9935eb711810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Nov 03 21:38:50.172533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:50.168880    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x4f2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Nov 03 21:38:50.172533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:50.168944    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g" podUID="ea097b76-14e5-48e9-9966-ed40754d66f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Nov 03 21:38:51.147838 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:51.147802    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lq9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Nov 03 21:38:51.148106 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:51.148080    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw" podUID="6174dd8c-e88b-48eb-8ba6-9935eb711810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Nov 03 21:38:51.164926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:51.164897    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x4f2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Nov 03 21:38:51.165142 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:51.165118    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g" podUID="ea097b76-14e5-48e9-9966-ed40754d66f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Nov 03 21:38:52.147327 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:52.147282    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lq9lw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Nov 03 21:38:52.147748 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:52.147347    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw" podUID="6174dd8c-e88b-48eb-8ba6-9935eb711810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Nov 03 21:38:52.169531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:52.164873    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x4f2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Nov 03 21:38:52.169531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:52.164929    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g" podUID="ea097b76-14e5-48e9-9966-ed40754d66f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Nov 03 21:38:52.371263 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:38:52.371222    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:38:53.151648 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:53.151618    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:53.152079 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:53.151855    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:53.152598 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:53.152582    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw"
Nov 03 21:38:53.168490 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:53.168464    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:53.168822 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:53.168805    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:53.169482 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:53.169468    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g"
Nov 03 21:38:57.795703 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:38:57.795667    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:02.988196 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:02.988155    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:08.242921 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:08.242883    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:08.351502 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:08.351456    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:22.370621 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:39:22.370517    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:39:26.022839 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.022803    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"]
Nov 03 21:39:26.026035 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.026016    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.056100 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.056068    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 03 21:39:26.056100 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.056088    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 03 21:39:26.056100 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.056085    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xpfzh\""
Nov 03 21:39:26.056577 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.056560    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Nov 03 21:39:26.057500 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.057482    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"]
Nov 03 21:39:26.114986 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.114958    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:26.126210 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.126183    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.126358 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.126243    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.126434 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.126410    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.126544 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.126503    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.126669 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.126569    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.126669 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.126644    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwzs2\" (UniqueName: \"kubernetes.io/projected/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-kube-api-access-lwzs2\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.227734 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.227703    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.227930 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.227745    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.227930 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.227776    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwzs2\" (UniqueName: \"kubernetes.io/projected/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-kube-api-access-lwzs2\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.227930 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.227813    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.227930 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.227866    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.227930 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.227898    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.228252 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.228230    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.228318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.228281    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.228318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.228302    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.230016 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.229996    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.230179 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.230164    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.247587 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.247562    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwzs2\" (UniqueName: \"kubernetes.io/projected/d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0-kube-api-access-lwzs2\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt\" (UID: \"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.339840 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.339769    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:26.642815 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:26.642785    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"]
Nov 03 21:39:26.645555 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:39:26.645489    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd06ae51f_4910_4aa8_aa9d_91fe3dfb49a0.slice/crio-f6511e0c75b54fc56e0468721287e525318470d5f8000503f4ded1e1d593b2f3 WatchSource:0}: Error finding container f6511e0c75b54fc56e0468721287e525318470d5f8000503f4ded1e1d593b2f3: Status 404 returned error can't find the container with id f6511e0c75b54fc56e0468721287e525318470d5f8000503f4ded1e1d593b2f3
Nov 03 21:39:27.208157 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:27.208109    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:39:27.379753 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:27.379709    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt" event={"ID":"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0","Type":"ContainerStarted","Data":"f6511e0c75b54fc56e0468721287e525318470d5f8000503f4ded1e1d593b2f3"}
Nov 03 21:39:28.239476 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:28.239447    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:28.390844 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:28.390808    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt" event={"ID":"d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0","Type":"ContainerStarted","Data":"f1c7b0ef59e4f523edaa2d8cdf5f52dfb1ab67c7fd72df595ff3dd3098126b3b"}
Nov 03 21:39:28.430420 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:28.430343    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt" podStartSLOduration=1.859229585 podStartE2EDuration="3.430327036s" totalImagesPullingTime="1.571097451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:39:28.429942954 +0000 UTC m=+577.308765294" watchObservedRunningTime="2026-11-03 21:39:28.430327036 +0000 UTC m=+577.309149321"
Nov 03 21:39:29.340946 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:29.340893    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:30.729250 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:30.729219    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:31.424993 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:31.424965    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:39:33.515266 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:33.515231    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:34.825846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:34.825804    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:39:35.705531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:35.704661    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:35.841410 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:35.841374    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"]
Nov 03 21:39:35.848949 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:35.848908    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:35.875297 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:35.875267    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"]
Nov 03 21:39:35.904474 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:35.904450    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Nov 03 21:39:35.905423 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:35.905310    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:35.948041 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:35.948005    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ad5a89a-2a2f-451b-8aae-e336df0ed106-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:35.948245 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:35.948063    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ad5a89a-2a2f-451b-8aae-e336df0ed106-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:35.948245 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:35.948108    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-649n7\" (UniqueName: \"kubernetes.io/projected/4ad5a89a-2a2f-451b-8aae-e336df0ed106-kube-api-access-649n7\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:35.948245 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:35.948147    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ad5a89a-2a2f-451b-8aae-e336df0ed106-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:35.948560 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:35.948294    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ad5a89a-2a2f-451b-8aae-e336df0ed106-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:35.948560 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:35.948348    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ad5a89a-2a2f-451b-8aae-e336df0ed106-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.058620 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.057217    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ad5a89a-2a2f-451b-8aae-e336df0ed106-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.058789 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.058673    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ad5a89a-2a2f-451b-8aae-e336df0ed106-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.058789 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.058733    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ad5a89a-2a2f-451b-8aae-e336df0ed106-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.059186 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.059139    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ad5a89a-2a2f-451b-8aae-e336df0ed106-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.059301 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.059258    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ad5a89a-2a2f-451b-8aae-e336df0ed106-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.059360 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.059323    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-649n7\" (UniqueName: \"kubernetes.io/projected/4ad5a89a-2a2f-451b-8aae-e336df0ed106-kube-api-access-649n7\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.059412 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.059382    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ad5a89a-2a2f-451b-8aae-e336df0ed106-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.059826 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.059803    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ad5a89a-2a2f-451b-8aae-e336df0ed106-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.060167 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.060149    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ad5a89a-2a2f-451b-8aae-e336df0ed106-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.065843 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.065823    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ad5a89a-2a2f-451b-8aae-e336df0ed106-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.082729 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.082698    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-649n7\" (UniqueName: \"kubernetes.io/projected/4ad5a89a-2a2f-451b-8aae-e336df0ed106-kube-api-access-649n7\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.086947 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.086917    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ad5a89a-2a2f-451b-8aae-e336df0ed106-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mwjk9\" (UID: \"4ad5a89a-2a2f-451b-8aae-e336df0ed106\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.195331 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.195294    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:36.340650 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.340567    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:36.352957 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.352931    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:36.459478 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.459453    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt"
Nov 03 21:39:36.486189 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:36.486168    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"]
Nov 03 21:39:36.488289 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:39:36.488267    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ad5a89a_2a2f_451b_8aae_e336df0ed106.slice/crio-75bf19bbd3c35d7390b8eff593d15366b897ad1de25e3cb2b9ebe49d0c3b8d93 WatchSource:0}: Error finding container 75bf19bbd3c35d7390b8eff593d15366b897ad1de25e3cb2b9ebe49d0c3b8d93: Status 404 returned error can't find the container with id 75bf19bbd3c35d7390b8eff593d15366b897ad1de25e3cb2b9ebe49d0c3b8d93
Nov 03 21:39:37.455544 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:37.455479    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9" event={"ID":"4ad5a89a-2a2f-451b-8aae-e336df0ed106","Type":"ContainerStarted","Data":"fd9f6fd61f6326771dfacc5a3df6d9c7f7f19729bd79cccfb1264029c5ceb2cb"}
Nov 03 21:39:37.456046 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:37.455562    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9" event={"ID":"4ad5a89a-2a2f-451b-8aae-e336df0ed106","Type":"ContainerStarted","Data":"75bf19bbd3c35d7390b8eff593d15366b897ad1de25e3cb2b9ebe49d0c3b8d93"}
Nov 03 21:39:37.499077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:37.499026    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9" podStartSLOduration=2.291932647 podStartE2EDuration="2.499012275s" totalImagesPullingTime="207.079628ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:39:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:39:37.497122454 +0000 UTC m=+586.375944738" watchObservedRunningTime="2026-11-03 21:39:37.499012275 +0000 UTC m=+586.377834585"
Nov 03 21:39:37.535747 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:37.535717    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:38.196024 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:38.195982    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:39.140158 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.140120    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-4kqqw"]
Nov 03 21:39:39.143599 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.143577    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.164521 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.164480    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-4kqqw"]
Nov 03 21:39:39.175553 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.175530    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Nov 03 21:39:39.196984 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.196957    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:39.213025 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.212998    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.213155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.213038    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-home\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.213155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.213076    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7rk7\" (UniqueName: \"kubernetes.io/projected/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-kube-api-access-j7rk7\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.213155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.213137    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.213258 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.213165    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.213258 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.213224    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.314182 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.314147    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.314374 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.314204    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-home\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.314374 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.314244    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7rk7\" (UniqueName: \"kubernetes.io/projected/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-kube-api-access-j7rk7\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.314374 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.314280    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.314374 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.314302    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.314374 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.314345    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.314755 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.314725    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-home\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.314755 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.314747    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.314928 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.314906    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.316569 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.316543    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.316749 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.316731    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.361221 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.361022    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7rk7\" (UniqueName: \"kubernetes.io/projected/77c0ac9e-c55d-4503-b2e6-2f29fb6f4454-kube-api-access-j7rk7\") pod \"echo-simulator-kserve-64b6f8f96c-4kqqw\" (UID: \"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.384612 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.384579    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:39:39.458628 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.458547    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:39.773886 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:39.773722    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-4kqqw"]
Nov 03 21:39:39.777013 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:39:39.776982    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c0ac9e_c55d_4503_b2e6_2f29fb6f4454.slice/crio-a77c94893d2f9f74ff64ca337aa7d6c4c054b1c9c079dd6462125c7304f2e58d WatchSource:0}: Error finding container a77c94893d2f9f74ff64ca337aa7d6c4c054b1c9c079dd6462125c7304f2e58d: Status 404 returned error can't find the container with id a77c94893d2f9f74ff64ca337aa7d6c4c054b1c9c079dd6462125c7304f2e58d
Nov 03 21:39:40.528531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:40.526782    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw" event={"ID":"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454","Type":"ContainerStarted","Data":"a77c94893d2f9f74ff64ca337aa7d6c4c054b1c9c079dd6462125c7304f2e58d"}
Nov 03 21:39:42.539150 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:42.539064    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw" event={"ID":"77c0ac9e-c55d-4503-b2e6-2f29fb6f4454","Type":"ContainerStarted","Data":"ca1b18d9c21d2ac6370aeb9cc2aab0142caf5fc32fef5ee4633ee263b5b7a857"}
Nov 03 21:39:42.582531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:42.582454    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw" podStartSLOduration=1.116109311 podStartE2EDuration="3.582439527s" totalImagesPullingTime="2.466330216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:39:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:39:42.580517862 +0000 UTC m=+591.459340138" watchObservedRunningTime="2026-11-03 21:39:42.582439527 +0000 UTC m=+591.461261851"
Nov 03 21:39:42.883014 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:42.882977    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:43.388551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:43.388143    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:39:43.459688 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:43.459660    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:44.096386 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.096349    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"]
Nov 03 21:39:44.100454 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.100430    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.122622 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.122594    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"]
Nov 03 21:39:44.131268 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.131241    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Nov 03 21:39:44.201749 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.201719    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dkb6\" (UniqueName: \"kubernetes.io/projected/f9a60514-d1ae-4072-b77a-9eb51ff05a08-kube-api-access-2dkb6\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.201920 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.201767    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9a60514-d1ae-4072-b77a-9eb51ff05a08-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.201965 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.201938    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9a60514-d1ae-4072-b77a-9eb51ff05a08-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.202012 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.201996    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9a60514-d1ae-4072-b77a-9eb51ff05a08-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.202055 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.202041    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9a60514-d1ae-4072-b77a-9eb51ff05a08-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.202109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.202096    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9a60514-d1ae-4072-b77a-9eb51ff05a08-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.219841 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.219793    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:44.303232 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.303197    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dkb6\" (UniqueName: \"kubernetes.io/projected/f9a60514-d1ae-4072-b77a-9eb51ff05a08-kube-api-access-2dkb6\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.303430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.303250    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9a60514-d1ae-4072-b77a-9eb51ff05a08-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.303430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.303294    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9a60514-d1ae-4072-b77a-9eb51ff05a08-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.303430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.303332    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9a60514-d1ae-4072-b77a-9eb51ff05a08-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.303430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.303368    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9a60514-d1ae-4072-b77a-9eb51ff05a08-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.303430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.303394    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9a60514-d1ae-4072-b77a-9eb51ff05a08-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.317284 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.312341    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9a60514-d1ae-4072-b77a-9eb51ff05a08-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.317284 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.312771    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9a60514-d1ae-4072-b77a-9eb51ff05a08-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.321807 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.321285    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9a60514-d1ae-4072-b77a-9eb51ff05a08-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.321807 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.321305    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9a60514-d1ae-4072-b77a-9eb51ff05a08-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.322816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.322771    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9a60514-d1ae-4072-b77a-9eb51ff05a08-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.327221 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.326936    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dkb6\" (UniqueName: \"kubernetes.io/projected/f9a60514-d1ae-4072-b77a-9eb51ff05a08-kube-api-access-2dkb6\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn\" (UID: \"f9a60514-d1ae-4072-b77a-9eb51ff05a08\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.415493 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.415413    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:44.713871 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:44.713846    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"]
Nov 03 21:39:44.715996 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:39:44.715965    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9a60514_d1ae_4072_b77a_9eb51ff05a08.slice/crio-bec69b91ef4fdd5806bde0ac60a160c6cb0d5dd9ff4447454856f74322e511d3 WatchSource:0}: Error finding container bec69b91ef4fdd5806bde0ac60a160c6cb0d5dd9ff4447454856f74322e511d3: Status 404 returned error can't find the container with id bec69b91ef4fdd5806bde0ac60a160c6cb0d5dd9ff4447454856f74322e511d3
Nov 03 21:39:45.566658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:45.565799    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" event={"ID":"f9a60514-d1ae-4072-b77a-9eb51ff05a08","Type":"ContainerStarted","Data":"50768ecaa740570787a94e20eed29f0621de9e9052bcbc2f4dd1576ba700b21f"}
Nov 03 21:39:45.566658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:45.565853    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" event={"ID":"f9a60514-d1ae-4072-b77a-9eb51ff05a08","Type":"ContainerStarted","Data":"bec69b91ef4fdd5806bde0ac60a160c6cb0d5dd9ff4447454856f74322e511d3"}
Nov 03 21:39:45.609771 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:45.609711    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podStartSLOduration=1.413541607 podStartE2EDuration="1.609693914s" totalImagesPullingTime="196.152307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:39:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:39:45.607582463 +0000 UTC m=+594.486404752" watchObservedRunningTime="2026-11-03 21:39:45.609693914 +0000 UTC m=+594.488516198"
Nov 03 21:39:46.195735 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:46.195700    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:46.208121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:46.208092    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:46.416567 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:46.416527    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:46.583816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:46.583729    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mwjk9"
Nov 03 21:39:49.459463 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:49.459421    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:49.471453 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:49.471427    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:49.600555 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:49.600529    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-4kqqw"
Nov 03 21:39:50.307451 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:50.307411    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:39:51.252554 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:51.252527    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:39:51.253120 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:51.253102    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:39:52.370209 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:39:52.370173    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:39:54.416275 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:54.416240    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:54.430411 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:54.430385    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:39:54.620425 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:39:54.620396    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:40:09.054632 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:09.054598    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66ccfd4b68-5hxfz"]
Nov 03 21:40:09.058623 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:09.058600    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66ccfd4b68-5hxfz"
Nov 03 21:40:09.076153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:09.076129    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66ccfd4b68-5hxfz"]
Nov 03 21:40:09.137830 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:09.137799    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv2p2\" (UniqueName: \"kubernetes.io/projected/4ca0d8d5-2b74-488a-949a-69b6d17d9d9a-kube-api-access-wv2p2\") pod \"authorino-66ccfd4b68-5hxfz\" (UID: \"4ca0d8d5-2b74-488a-949a-69b6d17d9d9a\") " pod="kuadrant-system/authorino-66ccfd4b68-5hxfz"
Nov 03 21:40:09.137994 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:09.137847    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4ca0d8d5-2b74-488a-949a-69b6d17d9d9a-tls-cert\") pod \"authorino-66ccfd4b68-5hxfz\" (UID: \"4ca0d8d5-2b74-488a-949a-69b6d17d9d9a\") " pod="kuadrant-system/authorino-66ccfd4b68-5hxfz"
Nov 03 21:40:09.238420 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:09.238385    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv2p2\" (UniqueName: \"kubernetes.io/projected/4ca0d8d5-2b74-488a-949a-69b6d17d9d9a-kube-api-access-wv2p2\") pod \"authorino-66ccfd4b68-5hxfz\" (UID: \"4ca0d8d5-2b74-488a-949a-69b6d17d9d9a\") " pod="kuadrant-system/authorino-66ccfd4b68-5hxfz"
Nov 03 21:40:09.238609 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:09.238465    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4ca0d8d5-2b74-488a-949a-69b6d17d9d9a-tls-cert\") pod \"authorino-66ccfd4b68-5hxfz\" (UID: \"4ca0d8d5-2b74-488a-949a-69b6d17d9d9a\") " pod="kuadrant-system/authorino-66ccfd4b68-5hxfz"
Nov 03 21:40:09.241025 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:09.241000    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4ca0d8d5-2b74-488a-949a-69b6d17d9d9a-tls-cert\") pod \"authorino-66ccfd4b68-5hxfz\" (UID: \"4ca0d8d5-2b74-488a-949a-69b6d17d9d9a\") " pod="kuadrant-system/authorino-66ccfd4b68-5hxfz"
Nov 03 21:40:09.255566 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:09.255543    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv2p2\" (UniqueName: \"kubernetes.io/projected/4ca0d8d5-2b74-488a-949a-69b6d17d9d9a-kube-api-access-wv2p2\") pod \"authorino-66ccfd4b68-5hxfz\" (UID: \"4ca0d8d5-2b74-488a-949a-69b6d17d9d9a\") " pod="kuadrant-system/authorino-66ccfd4b68-5hxfz"
Nov 03 21:40:09.369330 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:09.369298    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66ccfd4b68-5hxfz"
Nov 03 21:40:09.657236 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:09.657163    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66ccfd4b68-5hxfz"]
Nov 03 21:40:09.662695 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:40:09.662665    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ca0d8d5_2b74_488a_949a_69b6d17d9d9a.slice/crio-b05b72eaed895705ed31d63b3825411adbf02cdcd6eeac425d9fd75f7463f2ed WatchSource:0}: Error finding container b05b72eaed895705ed31d63b3825411adbf02cdcd6eeac425d9fd75f7463f2ed: Status 404 returned error can't find the container with id b05b72eaed895705ed31d63b3825411adbf02cdcd6eeac425d9fd75f7463f2ed
Nov 03 21:40:09.671822 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:09.671797    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66ccfd4b68-5hxfz" event={"ID":"4ca0d8d5-2b74-488a-949a-69b6d17d9d9a","Type":"ContainerStarted","Data":"b05b72eaed895705ed31d63b3825411adbf02cdcd6eeac425d9fd75f7463f2ed"}
Nov 03 21:40:10.684086 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:10.683213    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66ccfd4b68-5hxfz" event={"ID":"4ca0d8d5-2b74-488a-949a-69b6d17d9d9a","Type":"ContainerStarted","Data":"c656bbc35d3e17f3279c0c731d4981eb32f47a010c2fa8e0ef1a1d4df7f4335f"}
Nov 03 21:40:10.719351 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:10.719270    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-66ccfd4b68-5hxfz" podStartSLOduration=1.360999366 podStartE2EDuration="1.719253744s" totalImagesPullingTime="358.254378ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:40:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:40:10.71838271 +0000 UTC m=+619.597204987" watchObservedRunningTime="2026-11-03 21:40:10.719253744 +0000 UTC m=+619.598076029"
Nov 03 21:40:10.744906 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:10.744871    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86cfdb459f-dlpkg"]
Nov 03 21:40:10.745351 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:10.745314    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-86cfdb459f-dlpkg" podUID="f46f35a2-d883-4b2e-b198-0a6381704215" containerName="authorino" containerID="cri-o://127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df" gracePeriod=30
Nov 03 21:40:11.008191 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.008155    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86cfdb459f-dlpkg"
Nov 03 21:40:11.163656 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.162860    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f46f35a2-d883-4b2e-b198-0a6381704215-tls-cert\" (UniqueName: \"kubernetes.io/secret/f46f35a2-d883-4b2e-b198-0a6381704215-tls-cert\") pod \"f46f35a2-d883-4b2e-b198-0a6381704215\" (UID: \"f46f35a2-d883-4b2e-b198-0a6381704215\") "
Nov 03 21:40:11.163656 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.162945    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f46f35a2-d883-4b2e-b198-0a6381704215-kube-api-access-zlrhx\" (UniqueName: \"kubernetes.io/projected/f46f35a2-d883-4b2e-b198-0a6381704215-kube-api-access-zlrhx\") pod \"f46f35a2-d883-4b2e-b198-0a6381704215\" (UID: \"f46f35a2-d883-4b2e-b198-0a6381704215\") "
Nov 03 21:40:11.165810 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.165774    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f46f35a2-d883-4b2e-b198-0a6381704215-kube-api-access-zlrhx" pod "f46f35a2-d883-4b2e-b198-0a6381704215" (UID: "f46f35a2-d883-4b2e-b198-0a6381704215"). InnerVolumeSpecName "kube-api-access-zlrhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:40:11.173624 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.173594    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f46f35a2-d883-4b2e-b198-0a6381704215-tls-cert" pod "f46f35a2-d883-4b2e-b198-0a6381704215" (UID: "f46f35a2-d883-4b2e-b198-0a6381704215"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:40:11.264697 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.264666    4454 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f46f35a2-d883-4b2e-b198-0a6381704215-tls-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:40:11.264697 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.264698    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlrhx\" (UniqueName: \"kubernetes.io/projected/f46f35a2-d883-4b2e-b198-0a6381704215-kube-api-access-zlrhx\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:40:11.317065 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.317028    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:11.327996 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.327965    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-86c7698f44-8wtv7"]
Nov 03 21:40:11.328395 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.328377    4454 cpu_manager.go:414] "containerMap: removing container" podUID="f46f35a2-d883-4b2e-b198-0a6381704215" containerName="authorino"
Nov 03 21:40:11.328671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.328398    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f46f35a2-d883-4b2e-b198-0a6381704215" containerName="authorino"
Nov 03 21:40:11.328671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.328533    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="f46f35a2-d883-4b2e-b198-0a6381704215" containerName="authorino"
Nov 03 21:40:11.331764 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.331744    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7"
Nov 03 21:40:11.345332 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.345309    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Nov 03 21:40:11.345422 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.345354    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-db5l7\""
Nov 03 21:40:11.345489 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.345454    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 03 21:40:11.345489 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.345481    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 03 21:40:11.345600 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.345484    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 03 21:40:11.355241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.353447    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-86c7698f44-8wtv7"]
Nov 03 21:40:11.374740 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.374703    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:40:11.476902 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.476859    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rnnr\" (UniqueName: \"kubernetes.io/projected/fa7f8897-062f-4957-925b-6f8e2b694085-kube-api-access-2rnnr\") pod \"ai-gateway-controller-86c7698f44-8wtv7\" (UID: \"fa7f8897-062f-4957-925b-6f8e2b694085\") " pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7"
Nov 03 21:40:11.477063 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.476916    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7f8897-062f-4957-925b-6f8e2b694085-metrics-certs\") pod \"ai-gateway-controller-86c7698f44-8wtv7\" (UID: \"fa7f8897-062f-4957-925b-6f8e2b694085\") " pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7"
Nov 03 21:40:11.566210 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.566118    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:11.578546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.578489    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rnnr\" (UniqueName: \"kubernetes.io/projected/fa7f8897-062f-4957-925b-6f8e2b694085-kube-api-access-2rnnr\") pod \"ai-gateway-controller-86c7698f44-8wtv7\" (UID: \"fa7f8897-062f-4957-925b-6f8e2b694085\") " pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7"
Nov 03 21:40:11.578719 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.578566    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7f8897-062f-4957-925b-6f8e2b694085-metrics-certs\") pod \"ai-gateway-controller-86c7698f44-8wtv7\" (UID: \"fa7f8897-062f-4957-925b-6f8e2b694085\") " pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7"
Nov 03 21:40:11.581132 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.581106    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7f8897-062f-4957-925b-6f8e2b694085-metrics-certs\") pod \"ai-gateway-controller-86c7698f44-8wtv7\" (UID: \"fa7f8897-062f-4957-925b-6f8e2b694085\") " pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7"
Nov 03 21:40:11.597287 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.597254    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rnnr\" (UniqueName: \"kubernetes.io/projected/fa7f8897-062f-4957-925b-6f8e2b694085-kube-api-access-2rnnr\") pod \"ai-gateway-controller-86c7698f44-8wtv7\" (UID: \"fa7f8897-062f-4957-925b-6f8e2b694085\") " pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7"
Nov 03 21:40:11.647534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.647241    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7"
Nov 03 21:40:11.689607 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.689568    4454 generic.go:356] "Generic (PLEG): container finished" podID="f46f35a2-d883-4b2e-b198-0a6381704215" containerID="127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df" exitCode=0
Nov 03 21:40:11.690278 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.690257    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86cfdb459f-dlpkg"
Nov 03 21:40:11.690684 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.690660    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86cfdb459f-dlpkg" event={"ID":"f46f35a2-d883-4b2e-b198-0a6381704215","Type":"ContainerDied","Data":"127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df"}
Nov 03 21:40:11.690777 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.690699    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86cfdb459f-dlpkg" event={"ID":"f46f35a2-d883-4b2e-b198-0a6381704215","Type":"ContainerDied","Data":"6aa51ae8fdbb25895d0f1b8909fc7d97667969caf224e26010f91c898fe31028"}
Nov 03 21:40:11.690777 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.690721    4454 scope.go:122] "RemoveContainer" containerID="127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df"
Nov 03 21:40:11.705161 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.705096    4454 scope.go:122] "RemoveContainer" containerID="127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df"
Nov 03 21:40:11.705561 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:40:11.705536    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df\": container with ID starting with 127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df not found: ID does not exist" containerID="127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df"
Nov 03 21:40:11.705624 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.705574    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df"} err="failed to get container status \"127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df\": rpc error: code = NotFound desc = could not find container \"127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df\": container with ID starting with 127682c01c385b9066574d048a4e941ff16f7741f58111a0af3f1c77061df6df not found: ID does not exist"
Nov 03 21:40:11.739532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.738277    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86cfdb459f-dlpkg"]
Nov 03 21:40:11.762546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.757666    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-86cfdb459f-dlpkg"]
Nov 03 21:40:11.960088 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.960053    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-86c7698f44-8wtv7"]
Nov 03 21:40:11.965790 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:40:11.965670    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa7f8897_062f_4957_925b_6f8e2b694085.slice/crio-9645be540002a3625be69a283a775237a916069354dce080ef995bcb21db78f7 WatchSource:0}: Error finding container 9645be540002a3625be69a283a775237a916069354dce080ef995bcb21db78f7: Status 404 returned error can't find the container with id 9645be540002a3625be69a283a775237a916069354dce080ef995bcb21db78f7
Nov 03 21:40:11.969859 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:11.969304    4454 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:40:12.710643 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:12.710546    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7" event={"ID":"fa7f8897-062f-4957-925b-6f8e2b694085","Type":"ContainerStarted","Data":"9645be540002a3625be69a283a775237a916069354dce080ef995bcb21db78f7"}
Nov 03 21:40:13.254716 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:13.252904    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f46f35a2-d883-4b2e-b198-0a6381704215" path="/var/lib/kubelet/pods/f46f35a2-d883-4b2e-b198-0a6381704215/volumes"
Nov 03 21:40:16.743426 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:16.743385    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7" event={"ID":"fa7f8897-062f-4957-925b-6f8e2b694085","Type":"ContainerStarted","Data":"ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca"}
Nov 03 21:40:16.743839 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:16.743542    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7"
Nov 03 21:40:16.780020 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:16.779968    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7" podStartSLOduration=1.6162877469999999 podStartE2EDuration="5.779954736s" totalImagesPullingTime="4.163666989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:40:16.778202792 +0000 UTC m=+625.657025093" watchObservedRunningTime="2026-11-03 21:40:16.779954736 +0000 UTC m=+625.658777018"
Nov 03 21:40:18.365235 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.365195    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:18.467121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.467079    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-wgf6d"]
Nov 03 21:40:18.468570 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.468534    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:18.471138 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.471123    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-wgf6d"
Nov 03 21:40:18.492711 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.492686    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-wgf6d"]
Nov 03 21:40:18.500005 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.499980    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Nov 03 21:40:18.500118 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.500014    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4pxx4\""
Nov 03 21:40:18.596700 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.596663    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6845d56-0ca9-4dd3-a778-9849db797e23-plugins-config-volume\") pod \"payload-processing-7bb9489676-wgf6d\" (UID: \"b6845d56-0ca9-4dd3-a778-9849db797e23\") " pod="openshift-ingress/payload-processing-7bb9489676-wgf6d"
Nov 03 21:40:18.596896 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.596768    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25v9v\" (UniqueName: \"kubernetes.io/projected/b6845d56-0ca9-4dd3-a778-9849db797e23-kube-api-access-25v9v\") pod \"payload-processing-7bb9489676-wgf6d\" (UID: \"b6845d56-0ca9-4dd3-a778-9849db797e23\") " pod="openshift-ingress/payload-processing-7bb9489676-wgf6d"
Nov 03 21:40:18.697492 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.697411    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6845d56-0ca9-4dd3-a778-9849db797e23-plugins-config-volume\") pod \"payload-processing-7bb9489676-wgf6d\" (UID: \"b6845d56-0ca9-4dd3-a778-9849db797e23\") " pod="openshift-ingress/payload-processing-7bb9489676-wgf6d"
Nov 03 21:40:18.697492 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.697443    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25v9v\" (UniqueName: \"kubernetes.io/projected/b6845d56-0ca9-4dd3-a778-9849db797e23-kube-api-access-25v9v\") pod \"payload-processing-7bb9489676-wgf6d\" (UID: \"b6845d56-0ca9-4dd3-a778-9849db797e23\") " pod="openshift-ingress/payload-processing-7bb9489676-wgf6d"
Nov 03 21:40:18.698165 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.698144    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6845d56-0ca9-4dd3-a778-9849db797e23-plugins-config-volume\") pod \"payload-processing-7bb9489676-wgf6d\" (UID: \"b6845d56-0ca9-4dd3-a778-9849db797e23\") " pod="openshift-ingress/payload-processing-7bb9489676-wgf6d"
Nov 03 21:40:18.715208 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.715174    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25v9v\" (UniqueName: \"kubernetes.io/projected/b6845d56-0ca9-4dd3-a778-9849db797e23-kube-api-access-25v9v\") pod \"payload-processing-7bb9489676-wgf6d\" (UID: \"b6845d56-0ca9-4dd3-a778-9849db797e23\") " pod="openshift-ingress/payload-processing-7bb9489676-wgf6d"
Nov 03 21:40:18.783794 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:18.783766    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-wgf6d"
Nov 03 21:40:19.074237 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:19.074157    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-wgf6d"]
Nov 03 21:40:19.079289 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:40:19.079256    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6845d56_0ca9_4dd3_a778_9849db797e23.slice/crio-1f16237380fc965a7cae3923168198d4b5e5c07421f3a2923db9a09e770b83fc WatchSource:0}: Error finding container 1f16237380fc965a7cae3923168198d4b5e5c07421f3a2923db9a09e770b83fc: Status 404 returned error can't find the container with id 1f16237380fc965a7cae3923168198d4b5e5c07421f3a2923db9a09e770b83fc
Nov 03 21:40:19.767015 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:19.766972    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-wgf6d" event={"ID":"b6845d56-0ca9-4dd3-a778-9849db797e23","Type":"ContainerStarted","Data":"1f16237380fc965a7cae3923168198d4b5e5c07421f3a2923db9a09e770b83fc"}
Nov 03 21:40:21.778355 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:21.778320    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-wgf6d" event={"ID":"b6845d56-0ca9-4dd3-a778-9849db797e23","Type":"ContainerStarted","Data":"32e558694650018aa3ee1fe65ccca8a826b6da5282457b776960d8048b8e08cb"}
Nov 03 21:40:21.778796 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:21.778422    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bb9489676-wgf6d"
Nov 03 21:40:21.813877 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:21.813830    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bb9489676-wgf6d" podStartSLOduration=1.82991254 podStartE2EDuration="3.813818534s" totalImagesPullingTime="1.983905994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:40:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:40:21.812740854 +0000 UTC m=+630.691563158" watchObservedRunningTime="2026-11-03 21:40:21.813818534 +0000 UTC m=+630.692640847"
Nov 03 21:40:22.370404 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:40:22.370366    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:40:27.750858 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:27.750828    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7"
Nov 03 21:40:28.030223 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:28.030131    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"]
Nov 03 21:40:28.034048 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:28.034023    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"
Nov 03 21:40:28.052494 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:28.052473    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"]
Nov 03 21:40:28.187962 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:28.187921    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfz96\" (UniqueName: \"kubernetes.io/projected/54a67527-d872-42d9-9e96-fad468debfe8-kube-api-access-zfz96\") pod \"ai-gateway-controller-76cf67fc7b-5hk42\" (UID: \"54a67527-d872-42d9-9e96-fad468debfe8\") " pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"
Nov 03 21:40:28.187962 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:28.187966    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a67527-d872-42d9-9e96-fad468debfe8-metrics-certs\") pod \"ai-gateway-controller-76cf67fc7b-5hk42\" (UID: \"54a67527-d872-42d9-9e96-fad468debfe8\") " pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"
Nov 03 21:40:28.288910 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:28.288822    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfz96\" (UniqueName: \"kubernetes.io/projected/54a67527-d872-42d9-9e96-fad468debfe8-kube-api-access-zfz96\") pod \"ai-gateway-controller-76cf67fc7b-5hk42\" (UID: \"54a67527-d872-42d9-9e96-fad468debfe8\") " pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"
Nov 03 21:40:28.288910 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:28.288861    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a67527-d872-42d9-9e96-fad468debfe8-metrics-certs\") pod \"ai-gateway-controller-76cf67fc7b-5hk42\" (UID: \"54a67527-d872-42d9-9e96-fad468debfe8\") " pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"
Nov 03 21:40:28.291220 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:28.291195    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54a67527-d872-42d9-9e96-fad468debfe8-metrics-certs\") pod \"ai-gateway-controller-76cf67fc7b-5hk42\" (UID: \"54a67527-d872-42d9-9e96-fad468debfe8\") " pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"
Nov 03 21:40:28.306771 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:28.306749    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfz96\" (UniqueName: \"kubernetes.io/projected/54a67527-d872-42d9-9e96-fad468debfe8-kube-api-access-zfz96\") pod \"ai-gateway-controller-76cf67fc7b-5hk42\" (UID: \"54a67527-d872-42d9-9e96-fad468debfe8\") " pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"
Nov 03 21:40:28.346788 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:28.346761    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"
Nov 03 21:40:28.636191 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:28.636151    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"]
Nov 03 21:40:28.640236 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:40:28.640197    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a67527_d872_42d9_9e96_fad468debfe8.slice/crio-51db746f99cb0ff49df4fb355e56de9e2daf3ac46a3729bbab3fea9ab3b5110d WatchSource:0}: Error finding container 51db746f99cb0ff49df4fb355e56de9e2daf3ac46a3729bbab3fea9ab3b5110d: Status 404 returned error can't find the container with id 51db746f99cb0ff49df4fb355e56de9e2daf3ac46a3729bbab3fea9ab3b5110d
Nov 03 21:40:28.810109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:28.810077    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42" event={"ID":"54a67527-d872-42d9-9e96-fad468debfe8","Type":"ContainerStarted","Data":"51db746f99cb0ff49df4fb355e56de9e2daf3ac46a3729bbab3fea9ab3b5110d"}
Nov 03 21:40:29.815582 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:29.815545    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42" event={"ID":"54a67527-d872-42d9-9e96-fad468debfe8","Type":"ContainerStarted","Data":"ad997cc9195c4fea53e3055486bcc009fbea61119b7e3e652a7f3b104ff48025"}
Nov 03 21:40:29.815964 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:29.815690    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"
Nov 03 21:40:29.851017 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:29.850975    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42" podStartSLOduration=1.570286069 podStartE2EDuration="1.850962927s" totalImagesPullingTime="280.676858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:40:29.850235767 +0000 UTC m=+638.729058051" watchObservedRunningTime="2026-11-03 21:40:29.850962927 +0000 UTC m=+638.729785210"
Nov 03 21:40:32.785662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:32.785629    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bb9489676-wgf6d"
Nov 03 21:40:35.819001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:35.818963    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:40.822301 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:40.822269    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-76cf67fc7b-5hk42"
Nov 03 21:40:40.887465 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:40.887432    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-86c7698f44-8wtv7"]
Nov 03 21:40:40.888050 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:40.887723    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7" podUID="fa7f8897-062f-4957-925b-6f8e2b694085" containerName="manager" containerID="cri-o://ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca" gracePeriod=30
Nov 03 21:40:41.135463 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.135437    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7"
Nov 03 21:40:41.198500 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.198468    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa7f8897-062f-4957-925b-6f8e2b694085-kube-api-access-2rnnr\" (UniqueName: \"kubernetes.io/projected/fa7f8897-062f-4957-925b-6f8e2b694085-kube-api-access-2rnnr\") pod \"fa7f8897-062f-4957-925b-6f8e2b694085\" (UID: \"fa7f8897-062f-4957-925b-6f8e2b694085\") "
Nov 03 21:40:41.198670 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.198518    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa7f8897-062f-4957-925b-6f8e2b694085-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7f8897-062f-4957-925b-6f8e2b694085-metrics-certs\") pod \"fa7f8897-062f-4957-925b-6f8e2b694085\" (UID: \"fa7f8897-062f-4957-925b-6f8e2b694085\") "
Nov 03 21:40:41.200707 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.200674    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa7f8897-062f-4957-925b-6f8e2b694085-metrics-certs" pod "fa7f8897-062f-4957-925b-6f8e2b694085" (UID: "fa7f8897-062f-4957-925b-6f8e2b694085"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:40:41.200835 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.200773    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa7f8897-062f-4957-925b-6f8e2b694085-kube-api-access-2rnnr" pod "fa7f8897-062f-4957-925b-6f8e2b694085" (UID: "fa7f8897-062f-4957-925b-6f8e2b694085"). InnerVolumeSpecName "kube-api-access-2rnnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:40:41.299804 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.299773    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rnnr\" (UniqueName: \"kubernetes.io/projected/fa7f8897-062f-4957-925b-6f8e2b694085-kube-api-access-2rnnr\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:40:41.299804 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.299798    4454 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa7f8897-062f-4957-925b-6f8e2b694085-metrics-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:40:41.868263 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.868224    4454 generic.go:356] "Generic (PLEG): container finished" podID="fa7f8897-062f-4957-925b-6f8e2b694085" containerID="ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca" exitCode=0
Nov 03 21:40:41.868706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.868273    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7"
Nov 03 21:40:41.868706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.868303    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7" event={"ID":"fa7f8897-062f-4957-925b-6f8e2b694085","Type":"ContainerDied","Data":"ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca"}
Nov 03 21:40:41.868706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.868354    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-86c7698f44-8wtv7" event={"ID":"fa7f8897-062f-4957-925b-6f8e2b694085","Type":"ContainerDied","Data":"9645be540002a3625be69a283a775237a916069354dce080ef995bcb21db78f7"}
Nov 03 21:40:41.868706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.868376    4454 scope.go:122] "RemoveContainer" containerID="ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca"
Nov 03 21:40:41.876221 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.876206    4454 scope.go:122] "RemoveContainer" containerID="ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca"
Nov 03 21:40:41.876436 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:40:41.876421    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca\": container with ID starting with ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca not found: ID does not exist" containerID="ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca"
Nov 03 21:40:41.876475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.876444    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca"} err="failed to get container status \"ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca\": rpc error: code = NotFound desc = could not find container \"ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca\": container with ID starting with ecd5e4a729f99a8a73d2ec0aa3854b861840ebd15bba90c6c739bed8c1b94bca not found: ID does not exist"
Nov 03 21:40:41.913715 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.913690    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-86c7698f44-8wtv7"]
Nov 03 21:40:41.924605 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:41.924583    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-86c7698f44-8wtv7"]
Nov 03 21:40:42.574026 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:42.573993    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:43.098983 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:43.098951    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:43.228575 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:40:43.228543    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa7f8897-062f-4957-925b-6f8e2b694085" path="/var/lib/kubelet/pods/fa7f8897-062f-4957-925b-6f8e2b694085/volumes"
Nov 03 21:40:52.370304 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:40:52.370268    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:41:04.808714 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:04.808626    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:04.928003 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:04.927962    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:06.337156 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:06.337116    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:06.458284 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:06.458248    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:22.369370 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:41:22.369328    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:41:35.934702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:35.934660    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:38.781420 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:38.781349    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:38.879526 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:38.879467    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:39.275776 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.275742    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:39.335927 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.335899    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"]
Nov 03 21:41:39.336318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.336283    4454 cpu_manager.go:414] "containerMap: removing container" podUID="fa7f8897-062f-4957-925b-6f8e2b694085" containerName="manager"
Nov 03 21:41:39.336318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.336316    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa7f8897-062f-4957-925b-6f8e2b694085" containerName="manager"
Nov 03 21:41:39.336569 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.336449    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa7f8897-062f-4957-925b-6f8e2b694085" containerName="manager"
Nov 03 21:41:39.339773 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.339746    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.366448 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.366368    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"]
Nov 03 21:41:39.388953 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.388923    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:39.399107 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.399082    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-envoy\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.399249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.399124    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-podinfo\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.399249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.399141    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-token\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.399319 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.399273    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-certs\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.399356 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.399322    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istiod-ca-cert\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.399388 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.399374    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.399420 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.399400    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-data\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.399461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.399446    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-credential-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.399540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.399499    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8brkm\" (UniqueName: \"kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-kube-api-access-8brkm\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.411742 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.411722    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-4eced5-openshift-default-dockercfg-pf9dz\""
Nov 03 21:41:39.460619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.460593    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"]
Nov 03 21:41:39.465008 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.464985    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.479210 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.479188    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-e246f9-openshift-default-dockercfg-vn7bt\""
Nov 03 21:41:39.491187 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.491160    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"]
Nov 03 21:41:39.498318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.498286    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:39.500489 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500466    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istiod-ca-cert\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.500579 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500545    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-podinfo\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.500617 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500595    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-certs\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.500661 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500623    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-envoy\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.500661 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500651    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-credential-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.500745 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500668    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.500745 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500685    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-token\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.500846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500824    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-envoy\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.500898 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500876    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-podinfo\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.500898 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500895    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-token\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.501001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500919    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv2zs\" (UniqueName: \"kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-kube-api-access-pv2zs\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.501001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500953    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istiod-ca-cert\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.501001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.500978    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-data\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.501155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.501040    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.501155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.501064    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-data\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.501155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.501043    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-certs\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.501155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.501100    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-credential-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.501371 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.501164    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-certs\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.501371 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.501204    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8brkm\" (UniqueName: \"kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-kube-api-access-8brkm\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.501371 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.501316    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.501545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.501386    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-credential-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.501624 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.501603    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-data\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.501771 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.501751    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istiod-ca-cert\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.502992 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.502974    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-envoy\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.503225 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.503207    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-podinfo\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.518417 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.518389    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-token\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.518574 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.518486    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8brkm\" (UniqueName: \"kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-kube-api-access-8brkm\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.601877 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.601846    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv2zs\" (UniqueName: \"kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-kube-api-access-pv2zs\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.602053 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.601890    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-data\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.602053 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.601930    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-certs\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.602053 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.601967    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istiod-ca-cert\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.602053 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.602016    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-podinfo\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.602053 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.602050    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-envoy\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.602316 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.602092    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-credential-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.602316 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.602117    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.602316 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.602147    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-token\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.602316 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.602291    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-data\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.602528 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.602364    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-certs\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.602899 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.602734    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-credential-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.602899 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.602858    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.603061 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.602905    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istiod-ca-cert\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.605035 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.605013    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-podinfo\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.605192 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.605175    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-envoy\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.613132 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.613106    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:39.622398 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.622373    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-token\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.624232 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.624209    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv2zs\" (UniqueName: \"kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-kube-api-access-pv2zs\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.654312 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.654290    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:39.711784 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.711194    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"]
Nov 03 21:41:39.717126 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.717102    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.726393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.726360    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:39.738428 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.738404    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"]
Nov 03 21:41:39.746183 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.746156    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-0a6da1-openshift-default-dockercfg-dhxln\""
Nov 03 21:41:39.782746 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.782715    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:39.805103 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.805078    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-podinfo\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.805213 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.805108    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-certs\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.805213 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.805136    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-729ts\" (UniqueName: \"kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-kube-api-access-729ts\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.805294 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.805214    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istiod-ca-cert\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.805330 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.805295    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-data\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.805330 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.805325    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-envoy\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.805396 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.805347    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-socket\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.805396 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.805362    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-credential-socket\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.805462 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.805420    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-token\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.900609 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.900535    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:39.906799 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.906772    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-podinfo\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.906943 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.906805    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-certs\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.906943 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.906833    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-729ts\" (UniqueName: \"kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-kube-api-access-729ts\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.906943 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.906859    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istiod-ca-cert\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.906943 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.906898    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-data\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.906943 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.906931    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-envoy\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.907230 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.906966    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-socket\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.907230 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.906988    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-credential-socket\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.907230 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.907013    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-token\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.907230 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.907221    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-certs\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.907391 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.907230    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-data\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.907425 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.907385    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-socket\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.907425 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.907397    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-credential-socket\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.907802 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.907784    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istiod-ca-cert\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.909200 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.909179    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-envoy\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.909579 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.909563    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-podinfo\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.951079 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.950738    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-token\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.951079 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.951011    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-729ts\" (UniqueName: \"kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-kube-api-access-729ts\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:39.977017 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:39.976962    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"]
Nov 03 21:41:40.007200 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:41:40.007159    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc0ea14b_d4d9_4ab9_bfbe_aa260f33a515.slice/crio-e80493a7ee7dd09a04a53d799ea613bc5796a0d0e12f3e145ac4bf40288cc12e WatchSource:0}: Error finding container e80493a7ee7dd09a04a53d799ea613bc5796a0d0e12f3e145ac4bf40288cc12e: Status 404 returned error can't find the container with id e80493a7ee7dd09a04a53d799ea613bc5796a0d0e12f3e145ac4bf40288cc12e
Nov 03 21:41:40.009731 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.009694    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:41:40.009837 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.009769    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:41:40.009837 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.009799    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:41:40.028374 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.028343    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:40.038140 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.038119    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:40.080858 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.080809    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"]
Nov 03 21:41:40.086807 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:41:40.086779    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2588bcca_4842_455d_a4fd_7397fa0bdb5f.slice/crio-c37a0a46464905b77c5c2bb6b1f98facadb71506cd5672d37d560966887ea52f WatchSource:0}: Error finding container c37a0a46464905b77c5c2bb6b1f98facadb71506cd5672d37d560966887ea52f: Status 404 returned error can't find the container with id c37a0a46464905b77c5c2bb6b1f98facadb71506cd5672d37d560966887ea52f
Nov 03 21:41:40.089066 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.089034    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:41:40.089372 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.089099    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:41:40.089372 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.089130    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:41:40.147880 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.143152    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:40.147880 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.144568    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" event={"ID":"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515","Type":"ContainerStarted","Data":"ddecd189b98992a329b97f08595bf60f2970d964a525fb120b82ed9ce146b5e2"}
Nov 03 21:41:40.147880 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.144669    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" event={"ID":"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515","Type":"ContainerStarted","Data":"e80493a7ee7dd09a04a53d799ea613bc5796a0d0e12f3e145ac4bf40288cc12e"}
Nov 03 21:41:40.149750 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.149720    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" event={"ID":"2588bcca-4842-455d-a4fd-7397fa0bdb5f","Type":"ContainerStarted","Data":"c37a0a46464905b77c5c2bb6b1f98facadb71506cd5672d37d560966887ea52f"}
Nov 03 21:41:40.351173 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.351080    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" podStartSLOduration=1.35105903 podStartE2EDuration="1.35105903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:41:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:41:40.191686209 +0000 UTC m=+709.070508492" watchObservedRunningTime="2026-11-03 21:41:40.35105903 +0000 UTC m=+709.229881315"
Nov 03 21:41:40.352588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.352548    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"]
Nov 03 21:41:40.359958 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.359927    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:41:40.360068 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.359992    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:41:40.360068 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.360019    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:41:40.654684 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:40.654649    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:41.156073 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:41.156041    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" event={"ID":"b9a8e74a-6b9e-43df-b41e-20ecf8181814","Type":"ContainerStarted","Data":"021254eeddaafccc5fd66b10d434f48a69095d210d07373915b313da279876f2"}
Nov 03 21:41:41.156658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:41.156094    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" event={"ID":"b9a8e74a-6b9e-43df-b41e-20ecf8181814","Type":"ContainerStarted","Data":"ccfa0eae63d857ab3ecdc18a33448ae79a6faa6fa149cf1d6740679d8f977956"}
Nov 03 21:41:41.157600 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:41.157576    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" event={"ID":"2588bcca-4842-455d-a4fd-7397fa0bdb5f","Type":"ContainerStarted","Data":"b55135cf5db3afe449733f517d589fd3a5612d789265301840d3e24202895fba"}
Nov 03 21:41:41.218726 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:41.218669    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" podStartSLOduration=2.218652757 podStartE2EDuration="2.218652757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:41:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:41:41.205732101 +0000 UTC m=+710.084554404" watchObservedRunningTime="2026-11-03 21:41:41.218652757 +0000 UTC m=+710.097475044"
Nov 03 21:41:41.244737 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:41.244686    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" podStartSLOduration=2.244672685 podStartE2EDuration="2.244672685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:41:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:41:41.243323361 +0000 UTC m=+710.122145645" watchObservedRunningTime="2026-11-03 21:41:41.244672685 +0000 UTC m=+710.123494968"
Nov 03 21:41:41.660237 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:41.660208    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:41.783128 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:41.783091    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:41.787948 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:41.787923    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:42.038458 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:42.038373    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:42.043975 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:42.043953    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:42.163330 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:42.163298    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:42.163814 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:42.163354    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:42.163814 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:42.163379    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:42.164082 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:42.164056    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:41:42.164185 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:42.164124    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:41:42.164365 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:42.164344    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:41:42.676105 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:42.676071    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:42.781245 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:42.781196    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:44.332749 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.332709    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:44.417745 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.417711    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"]
Nov 03 21:41:44.421024 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.421001    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"
Nov 03 21:41:44.442948 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.442920    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"]
Nov 03 21:41:44.455836 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.455346    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-4eced5\""
Nov 03 21:41:44.455836 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.455548    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-4eced5-dockercfg-t2k4b\""
Nov 03 21:41:44.455836 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.455755    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.456159 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.456092    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b922f440-4072-4cfd-8cb1-f1a3646daaca-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f\" (UID: \"b922f440-4072-4cfd-8cb1-f1a3646daaca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"
Nov 03 21:41:44.456265 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.456233    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99smx\" (UniqueName: \"kubernetes.io/projected/b922f440-4072-4cfd-8cb1-f1a3646daaca-kube-api-access-99smx\") pod \"payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f\" (UID: \"b922f440-4072-4cfd-8cb1-f1a3646daaca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"
Nov 03 21:41:44.459088 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.459061    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"]
Nov 03 21:41:44.478127 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.478105    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-4eced5-serving-cert\""
Nov 03 21:41:44.486282 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.486257    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"]
Nov 03 21:41:44.490211 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.490188    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:44.556759 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.556730    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99af0d0b-bd91-4cc1-be41-d4414e335fac-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.556913 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.556775    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn92t\" (UniqueName: \"kubernetes.io/projected/99af0d0b-bd91-4cc1-be41-d4414e335fac-kube-api-access-pn92t\") pod \"maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.556913 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.556857    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99smx\" (UniqueName: \"kubernetes.io/projected/b922f440-4072-4cfd-8cb1-f1a3646daaca-kube-api-access-99smx\") pod \"payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f\" (UID: \"b922f440-4072-4cfd-8cb1-f1a3646daaca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"
Nov 03 21:41:44.556913 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.556888    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-maas-api-tls\") pod \"maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.557067 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.556936    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-metrics-certs\") pod \"maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.557067 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.556958    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b922f440-4072-4cfd-8cb1-f1a3646daaca-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f\" (UID: \"b922f440-4072-4cfd-8cb1-f1a3646daaca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"
Nov 03 21:41:44.557671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.557654    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b922f440-4072-4cfd-8cb1-f1a3646daaca-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f\" (UID: \"b922f440-4072-4cfd-8cb1-f1a3646daaca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"
Nov 03 21:41:44.575081 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.575058    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99smx\" (UniqueName: \"kubernetes.io/projected/b922f440-4072-4cfd-8cb1-f1a3646daaca-kube-api-access-99smx\") pod \"payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f\" (UID: \"b922f440-4072-4cfd-8cb1-f1a3646daaca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"
Nov 03 21:41:44.657791 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.657760    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99af0d0b-bd91-4cc1-be41-d4414e335fac-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.657964 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.657807    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn92t\" (UniqueName: \"kubernetes.io/projected/99af0d0b-bd91-4cc1-be41-d4414e335fac-kube-api-access-pn92t\") pod \"maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.657964 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.657833    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-maas-api-tls\") pod \"maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.657964 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.657901    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-metrics-certs\") pod \"maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.658648 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.658621    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99af0d0b-bd91-4cc1-be41-d4414e335fac-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.660522 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.660485    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-maas-api-tls\") pod \"maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.660688 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.660667    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-metrics-certs\") pod \"maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.679170 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.679146    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn92t\" (UniqueName: \"kubernetes.io/projected/99af0d0b-bd91-4cc1-be41-d4414e335fac-kube-api-access-pn92t\") pod \"maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.718789 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.718753    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:44.736216 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.736194    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"
Nov 03 21:41:44.773437 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.773397    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:44.836581 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:44.829361    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:45.028868 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:45.028825    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"]
Nov 03 21:41:45.033429 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:41:45.033399    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb922f440_4072_4cfd_8cb1_f1a3646daaca.slice/crio-29813e0fc3b69a1948e7367827edfe65d9330d56de423ca31dfaf7e4fde2ad09 WatchSource:0}: Error finding container 29813e0fc3b69a1948e7367827edfe65d9330d56de423ca31dfaf7e4fde2ad09: Status 404 returned error can't find the container with id 29813e0fc3b69a1948e7367827edfe65d9330d56de423ca31dfaf7e4fde2ad09
Nov 03 21:41:45.143465 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:45.143432    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:45.155862 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:45.155836    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"]
Nov 03 21:41:45.159810 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:41:45.159776    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99af0d0b_bd91_4cc1_be41_d4414e335fac.slice/crio-04f8843aaf1f958991fb94b2140e9c21b77a9bbaad8f8f5f6e8c28967f8378fe WatchSource:0}: Error finding container 04f8843aaf1f958991fb94b2140e9c21b77a9bbaad8f8f5f6e8c28967f8378fe: Status 404 returned error can't find the container with id 04f8843aaf1f958991fb94b2140e9c21b77a9bbaad8f8f5f6e8c28967f8378fe
Nov 03 21:41:45.177078 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:45.177054    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f" event={"ID":"b922f440-4072-4cfd-8cb1-f1a3646daaca","Type":"ContainerStarted","Data":"1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538"}
Nov 03 21:41:45.177218 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:45.177099    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f" event={"ID":"b922f440-4072-4cfd-8cb1-f1a3646daaca","Type":"ContainerStarted","Data":"29813e0fc3b69a1948e7367827edfe65d9330d56de423ca31dfaf7e4fde2ad09"}
Nov 03 21:41:45.177218 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:45.177146    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"
Nov 03 21:41:45.178248 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:45.178227    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv" event={"ID":"99af0d0b-bd91-4cc1-be41-d4414e335fac","Type":"ContainerStarted","Data":"04f8843aaf1f958991fb94b2140e9c21b77a9bbaad8f8f5f6e8c28967f8378fe"}
Nov 03 21:41:45.216188 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:45.216145    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f" podStartSLOduration=1.216131096 podStartE2EDuration="1.216131096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:41:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:41:45.215642386 +0000 UTC m=+714.094464670" watchObservedRunningTime="2026-11-03 21:41:45.216131096 +0000 UTC m=+714.094953379"
Nov 03 21:41:45.270853 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:45.270813    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:45.396737 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:45.396666    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:45.495750 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:45.495671    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:46.103485 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.103443    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:46.184073 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.184031    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv" event={"ID":"99af0d0b-bd91-4cc1-be41-d4414e335fac","Type":"ContainerStarted","Data":"62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9"}
Nov 03 21:41:46.184267 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.184126    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:46.186551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.186522    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"]
Nov 03 21:41:46.190401 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.190382    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"
Nov 03 21:41:46.204848 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.204829    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-27fdf4\""
Nov 03 21:41:46.205071 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.205047    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkrch\" (UniqueName: \"kubernetes.io/projected/e376c103-f66d-4082-a27a-09bc2c48557a-kube-api-access-qkrch\") pod \"payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm\" (UID: \"e376c103-f66d-4082-a27a-09bc2c48557a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"
Nov 03 21:41:46.205241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.205219    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e376c103-f66d-4082-a27a-09bc2c48557a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm\" (UID: \"e376c103-f66d-4082-a27a-09bc2c48557a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"
Nov 03 21:41:46.212577 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.212548    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"]
Nov 03 21:41:46.216476 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.216455    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"]
Nov 03 21:41:46.216596 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.216568    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"
Nov 03 21:41:46.220703 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.220677    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:46.226103 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.226057    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv" podStartSLOduration=1.876705504 podStartE2EDuration="2.22604422s" totalImagesPullingTime="349.338716ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:41:46.22248197 +0000 UTC m=+715.101304255" watchObservedRunningTime="2026-11-03 21:41:46.22604422 +0000 UTC m=+715.104866503"
Nov 03 21:41:46.231694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.231674    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-27fdf4-dockercfg-zq2q2\""
Nov 03 21:41:46.237630 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.237608    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"]
Nov 03 21:41:46.305841 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.305808    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkrch\" (UniqueName: \"kubernetes.io/projected/e376c103-f66d-4082-a27a-09bc2c48557a-kube-api-access-qkrch\") pod \"payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm\" (UID: \"e376c103-f66d-4082-a27a-09bc2c48557a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"
Nov 03 21:41:46.305841 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.305840    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e376c103-f66d-4082-a27a-09bc2c48557a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm\" (UID: \"e376c103-f66d-4082-a27a-09bc2c48557a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"
Nov 03 21:41:46.306680 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.306660    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e376c103-f66d-4082-a27a-09bc2c48557a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm\" (UID: \"e376c103-f66d-4082-a27a-09bc2c48557a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"
Nov 03 21:41:46.325099 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.325073    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkrch\" (UniqueName: \"kubernetes.io/projected/e376c103-f66d-4082-a27a-09bc2c48557a-kube-api-access-qkrch\") pod \"payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm\" (UID: \"e376c103-f66d-4082-a27a-09bc2c48557a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"
Nov 03 21:41:46.408071 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.407958    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8\" (UID: \"d6ab1abf-4dd9-43d2-845e-225243e8d9d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"
Nov 03 21:41:46.408071 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.408042    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsjgz\" (UniqueName: \"kubernetes.io/projected/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-kube-api-access-vsjgz\") pod \"payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8\" (UID: \"d6ab1abf-4dd9-43d2-845e-225243e8d9d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"
Nov 03 21:41:46.503299 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.503267    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"
Nov 03 21:41:46.508690 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.508666    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8\" (UID: \"d6ab1abf-4dd9-43d2-845e-225243e8d9d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"
Nov 03 21:41:46.508782 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.508704    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsjgz\" (UniqueName: \"kubernetes.io/projected/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-kube-api-access-vsjgz\") pod \"payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8\" (UID: \"d6ab1abf-4dd9-43d2-845e-225243e8d9d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"
Nov 03 21:41:46.509542 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.509497    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8\" (UID: \"d6ab1abf-4dd9-43d2-845e-225243e8d9d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"
Nov 03 21:41:46.548578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.548411    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsjgz\" (UniqueName: \"kubernetes.io/projected/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-kube-api-access-vsjgz\") pod \"payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8\" (UID: \"d6ab1abf-4dd9-43d2-845e-225243e8d9d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"
Nov 03 21:41:46.717825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.717743    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:46.793980 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.793945    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"]
Nov 03 21:41:46.798416 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:41:46.798390    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode376c103_f66d_4082_a27a_09bc2c48557a.slice/crio-48f1726d26f926e17e4a0f15eb93e52a11ba3277c84667729a1c8d43b8d7246b WatchSource:0}: Error finding container 48f1726d26f926e17e4a0f15eb93e52a11ba3277c84667729a1c8d43b8d7246b: Status 404 returned error can't find the container with id 48f1726d26f926e17e4a0f15eb93e52a11ba3277c84667729a1c8d43b8d7246b
Nov 03 21:41:46.818848 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.818820    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:46.833214 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:46.833194    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"
Nov 03 21:41:47.150385 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:47.150309    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"]
Nov 03 21:41:47.154321 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:41:47.154295    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ab1abf_4dd9_43d2_845e_225243e8d9d6.slice/crio-f72832122f78d44cf7b16404e8f867b04673883c922993c3436194a529548e4e WatchSource:0}: Error finding container f72832122f78d44cf7b16404e8f867b04673883c922993c3436194a529548e4e: Status 404 returned error can't find the container with id f72832122f78d44cf7b16404e8f867b04673883c922993c3436194a529548e4e
Nov 03 21:41:47.194274 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:47.194230    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8" event={"ID":"d6ab1abf-4dd9-43d2-845e-225243e8d9d6","Type":"ContainerStarted","Data":"f72832122f78d44cf7b16404e8f867b04673883c922993c3436194a529548e4e"}
Nov 03 21:41:47.197553 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:47.196946    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm" event={"ID":"e376c103-f66d-4082-a27a-09bc2c48557a","Type":"ContainerStarted","Data":"a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32"}
Nov 03 21:41:47.197553 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:47.197015    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm" event={"ID":"e376c103-f66d-4082-a27a-09bc2c48557a","Type":"ContainerStarted","Data":"48f1726d26f926e17e4a0f15eb93e52a11ba3277c84667729a1c8d43b8d7246b"}
Nov 03 21:41:47.198006 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:47.197950    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"
Nov 03 21:41:47.236909 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:47.235589    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm" podStartSLOduration=1.235570702 podStartE2EDuration="1.235570702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:41:47.234175782 +0000 UTC m=+716.112998086" watchObservedRunningTime="2026-11-03 21:41:47.235570702 +0000 UTC m=+716.114392989"
Nov 03 21:41:48.169360 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:48.169326    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:48.203052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:48.203021    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8" event={"ID":"d6ab1abf-4dd9-43d2-845e-225243e8d9d6","Type":"ContainerStarted","Data":"5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34"}
Nov 03 21:41:48.203283 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:48.203267    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"
Nov 03 21:41:48.244524 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:48.244473    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8" podStartSLOduration=2.244460689 podStartE2EDuration="2.244460689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:41:48.243394087 +0000 UTC m=+717.122216370" watchObservedRunningTime="2026-11-03 21:41:48.244460689 +0000 UTC m=+717.123282972"
Nov 03 21:41:48.322176 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:48.322129    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:48.615361 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:48.615324    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:41:48.643375 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:48.643333    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:48.765641 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:48.765603    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:48.974532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:48.974426    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:49.060691 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.060655    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"]
Nov 03 21:41:49.064896 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.064872    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"
Nov 03 21:41:49.080253 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.080225    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:49.082314 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.082291    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-dd5620\""
Nov 03 21:41:49.095989 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.095966    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"]
Nov 03 21:41:49.141691 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.141658    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8spsk\" (UniqueName: \"kubernetes.io/projected/68b43a4e-3607-4620-b7a6-baf9c35cbf15-kube-api-access-8spsk\") pod \"payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v\" (UID: \"68b43a4e-3607-4620-b7a6-baf9c35cbf15\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"
Nov 03 21:41:49.141856 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.141719    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68b43a4e-3607-4620-b7a6-baf9c35cbf15-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v\" (UID: \"68b43a4e-3607-4620-b7a6-baf9c35cbf15\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"
Nov 03 21:41:49.231694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.231598    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:49.242934 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.242910    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8spsk\" (UniqueName: \"kubernetes.io/projected/68b43a4e-3607-4620-b7a6-baf9c35cbf15-kube-api-access-8spsk\") pod \"payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v\" (UID: \"68b43a4e-3607-4620-b7a6-baf9c35cbf15\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"
Nov 03 21:41:49.243067 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.242961    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68b43a4e-3607-4620-b7a6-baf9c35cbf15-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v\" (UID: \"68b43a4e-3607-4620-b7a6-baf9c35cbf15\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"
Nov 03 21:41:49.243837 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.243811    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68b43a4e-3607-4620-b7a6-baf9c35cbf15-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v\" (UID: \"68b43a4e-3607-4620-b7a6-baf9c35cbf15\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"
Nov 03 21:41:49.260651 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.260629    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8spsk\" (UniqueName: \"kubernetes.io/projected/68b43a4e-3607-4620-b7a6-baf9c35cbf15-kube-api-access-8spsk\") pod \"payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v\" (UID: \"68b43a4e-3607-4620-b7a6-baf9c35cbf15\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"
Nov 03 21:41:49.379280 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.379243    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"
Nov 03 21:41:49.411691 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.411662    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:49.674958 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:49.674916    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"]
Nov 03 21:41:49.694122 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:41:49.694090    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68b43a4e_3607_4620_b7a6_baf9c35cbf15.slice/crio-6e6abbcaf6d11d1c4c963a789e0a42413463e5729ea51f957423e7eceede84ef WatchSource:0}: Error finding container 6e6abbcaf6d11d1c4c963a789e0a42413463e5729ea51f957423e7eceede84ef: Status 404 returned error can't find the container with id 6e6abbcaf6d11d1c4c963a789e0a42413463e5729ea51f957423e7eceede84ef
Nov 03 21:41:50.007526 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.007400    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:50.223886 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.223848    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v" event={"ID":"68b43a4e-3607-4620-b7a6-baf9c35cbf15","Type":"ContainerStarted","Data":"42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42"}
Nov 03 21:41:50.224065 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.223904    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v" event={"ID":"68b43a4e-3607-4620-b7a6-baf9c35cbf15","Type":"ContainerStarted","Data":"6e6abbcaf6d11d1c4c963a789e0a42413463e5729ea51f957423e7eceede84ef"}
Nov 03 21:41:50.224065 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.223932    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"
Nov 03 21:41:50.254307 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.254275    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"]
Nov 03 21:41:50.257792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.257742    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.283530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.280433    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-27fdf4-serving-cert\""
Nov 03 21:41:50.283530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.280982    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v" podStartSLOduration=1.280967808 podStartE2EDuration="1.280967808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:41:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:41:50.279999626 +0000 UTC m=+719.158821924" watchObservedRunningTime="2026-11-03 21:41:50.280967808 +0000 UTC m=+719.159790094"
Nov 03 21:41:50.296690 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.296477    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"]
Nov 03 21:41:50.296690 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.296599    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:50.356451 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.356421    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-maas-api-tls\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.356451 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.356454    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxq6j\" (UniqueName: \"kubernetes.io/projected/b35f8fb8-4b99-48ac-88d1-81975b9c655e-kube-api-access-qxq6j\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.356687 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.356557    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35f8fb8-4b99-48ac-88d1-81975b9c655e-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.356687 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.356605    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-metrics-certs\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.389547 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.389490    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:50.437753 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.437719    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"]
Nov 03 21:41:50.441714 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.441684    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"
Nov 03 21:41:50.458046 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.458019    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-maas-api-tls\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.458183 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.458054    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxq6j\" (UniqueName: \"kubernetes.io/projected/b35f8fb8-4b99-48ac-88d1-81975b9c655e-kube-api-access-qxq6j\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.458183 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.458094    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35f8fb8-4b99-48ac-88d1-81975b9c655e-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.458183 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.458137    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-metrics-certs\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.458183 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.458171    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33553b17-5b86-4d27-91d7-1d0d29a75280-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2\" (UID: \"33553b17-5b86-4d27-91d7-1d0d29a75280\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"
Nov 03 21:41:50.458421 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:41:50.458196    4454 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-serving-cert: secret "maas-api-e2e-worker-w2-27fdf4-serving-cert" not found
Nov 03 21:41:50.458421 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.458206    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqz2g\" (UniqueName: \"kubernetes.io/projected/33553b17-5b86-4d27-91d7-1d0d29a75280-kube-api-access-jqz2g\") pod \"payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2\" (UID: \"33553b17-5b86-4d27-91d7-1d0d29a75280\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"
Nov 03 21:41:50.458421 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:41:50.458268    4454 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-maas-api-tls podName:b35f8fb8-4b99-48ac-88d1-81975b9c655e nodeName:}" failed. No retries permitted until 2026-11-03 21:41:50.958246072 +0000 UTC m=+719.837068352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-maas-api-tls") pod "maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l" (UID: "b35f8fb8-4b99-48ac-88d1-81975b9c655e") : secret "maas-api-e2e-worker-w2-27fdf4-serving-cert" not found
Nov 03 21:41:50.458994 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.458974    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35f8fb8-4b99-48ac-88d1-81975b9c655e-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.482758 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.482709    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"]
Nov 03 21:41:50.493875 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.485777    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-metrics-certs\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.501267 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.501230    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxq6j\" (UniqueName: \"kubernetes.io/projected/b35f8fb8-4b99-48ac-88d1-81975b9c655e-kube-api-access-qxq6j\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.502759 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.502724    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:50.517599 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.516826    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-05bc27-dockercfg-2bz6d\""
Nov 03 21:41:50.535625 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.517910    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-05bc27\""
Nov 03 21:41:50.579529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.559605    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33553b17-5b86-4d27-91d7-1d0d29a75280-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2\" (UID: \"33553b17-5b86-4d27-91d7-1d0d29a75280\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"
Nov 03 21:41:50.579529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.559662    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqz2g\" (UniqueName: \"kubernetes.io/projected/33553b17-5b86-4d27-91d7-1d0d29a75280-kube-api-access-jqz2g\") pod \"payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2\" (UID: \"33553b17-5b86-4d27-91d7-1d0d29a75280\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"
Nov 03 21:41:50.579529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.560971    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33553b17-5b86-4d27-91d7-1d0d29a75280-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2\" (UID: \"33553b17-5b86-4d27-91d7-1d0d29a75280\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"
Nov 03 21:41:50.614551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.613647    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqz2g\" (UniqueName: \"kubernetes.io/projected/33553b17-5b86-4d27-91d7-1d0d29a75280-kube-api-access-jqz2g\") pod \"payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2\" (UID: \"33553b17-5b86-4d27-91d7-1d0d29a75280\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"
Nov 03 21:41:50.632362 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.632327    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:50.755311 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.755276    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"
Nov 03 21:41:50.985534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.985188    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-maas-api-tls\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:50.987837 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:50.987813    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-maas-api-tls\") pod \"maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:51.089560 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:51.088151    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"]
Nov 03 21:41:51.204380 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:51.204350    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:51.236586 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:51.236546    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2" event={"ID":"33553b17-5b86-4d27-91d7-1d0d29a75280","Type":"ContainerStarted","Data":"8a24fabef0f083d5e7809d8499b0c1eb3c055247a8133ee740af6d18f3b949e5"}
Nov 03 21:41:51.236733 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:51.236623    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2" event={"ID":"33553b17-5b86-4d27-91d7-1d0d29a75280","Type":"ContainerStarted","Data":"55d267f3b8f8e152f7bb51c57c3931662474e4656314d2449f9510ba1ada9ed8"}
Nov 03 21:41:51.236862 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:51.236837    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"
Nov 03 21:41:51.290542 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:51.289966    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2" podStartSLOduration=1.289947483 podStartE2EDuration="1.289947483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:41:51.289490817 +0000 UTC m=+720.168313103" watchObservedRunningTime="2026-11-03 21:41:51.289947483 +0000 UTC m=+720.168769771"
Nov 03 21:41:51.539099 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:51.539069    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"]
Nov 03 21:41:51.542690 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:41:51.542660    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb35f8fb8_4b99_48ac_88d1_81975b9c655e.slice/crio-a1fe3c1cd934a4dc9e7a795163dba5a304aac78cb1f4d5dae8d9d1301471c1aa WatchSource:0}: Error finding container a1fe3c1cd934a4dc9e7a795163dba5a304aac78cb1f4d5dae8d9d1301471c1aa: Status 404 returned error can't find the container with id a1fe3c1cd934a4dc9e7a795163dba5a304aac78cb1f4d5dae8d9d1301471c1aa
Nov 03 21:41:52.203052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.202979    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:41:52.218291 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.218260    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"]
Nov 03 21:41:52.222042 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.222021    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.235401 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.235381    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-0a6da1-serving-cert\""
Nov 03 21:41:52.244883 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.244856    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l" event={"ID":"b35f8fb8-4b99-48ac-88d1-81975b9c655e","Type":"ContainerStarted","Data":"c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635"}
Nov 03 21:41:52.245025 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.244910    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l" event={"ID":"b35f8fb8-4b99-48ac-88d1-81975b9c655e","Type":"ContainerStarted","Data":"a1fe3c1cd934a4dc9e7a795163dba5a304aac78cb1f4d5dae8d9d1301471c1aa"}
Nov 03 21:41:52.245025 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.244938    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:52.247548 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.247523    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"]
Nov 03 21:41:52.306105 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.306069    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12ae6e3f-92fb-43f4-9031-c91f75423aa4-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.306611 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.306242    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knnhq\" (UniqueName: \"kubernetes.io/projected/12ae6e3f-92fb-43f4-9031-c91f75423aa4-kube-api-access-knnhq\") pod \"maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.306611 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.306331    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-maas-api-tls\") pod \"maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.306738 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.306708    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-metrics-certs\") pod \"maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.310583 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.310554    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:52.333262 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.332855    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l" podStartSLOduration=1.957600668 podStartE2EDuration="2.332840657s" totalImagesPullingTime="375.239989ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:41:52.331219205 +0000 UTC m=+721.210041490" watchObservedRunningTime="2026-11-03 21:41:52.332840657 +0000 UTC m=+721.211662942"
Nov 03 21:41:52.394075 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:41:52.393971    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:41:52.407565 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.407495    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12ae6e3f-92fb-43f4-9031-c91f75423aa4-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.407753 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.407606    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knnhq\" (UniqueName: \"kubernetes.io/projected/12ae6e3f-92fb-43f4-9031-c91f75423aa4-kube-api-access-knnhq\") pod \"maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.407753 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.407636    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-maas-api-tls\") pod \"maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.407753 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.407703    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-metrics-certs\") pod \"maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.408284 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.408254    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12ae6e3f-92fb-43f4-9031-c91f75423aa4-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.408428 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.408388    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:52.410171 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.410151    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-maas-api-tls\") pod \"maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.410280 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.410183    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-metrics-certs\") pod \"maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.425486 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.425458    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knnhq\" (UniqueName: \"kubernetes.io/projected/12ae6e3f-92fb-43f4-9031-c91f75423aa4-kube-api-access-knnhq\") pod \"maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.520627 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.519966    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:52.534909 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.534879    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:52.787981 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.787158    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:52.887011 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.872221    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"]
Nov 03 21:41:52.888221 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:41:52.888190    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ae6e3f_92fb_43f4_9031_c91f75423aa4.slice/crio-5bb7288807444528649727afe4b82449367d8368e23338326539ade949c56dbd WatchSource:0}: Error finding container 5bb7288807444528649727afe4b82449367d8368e23338326539ade949c56dbd: Status 404 returned error can't find the container with id 5bb7288807444528649727afe4b82449367d8368e23338326539ade949c56dbd
Nov 03 21:41:52.986915 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:52.986881    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:53.250427 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:53.250392    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb" event={"ID":"12ae6e3f-92fb-43f4-9031-c91f75423aa4","Type":"ContainerStarted","Data":"5bb7288807444528649727afe4b82449367d8368e23338326539ade949c56dbd"}
Nov 03 21:41:53.403544 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:53.391882    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:41:53.749112 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:53.720141    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:53.831671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:53.831578    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:54.257628 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:54.257589    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb" event={"ID":"12ae6e3f-92fb-43f4-9031-c91f75423aa4","Type":"ContainerStarted","Data":"2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0"}
Nov 03 21:41:54.257806 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:54.257715    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:41:54.318600 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:54.318552    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb" podStartSLOduration=1.6266170899999999 podStartE2EDuration="2.318538365s" totalImagesPullingTime="691.921275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:41:54.318204258 +0000 UTC m=+723.197026541" watchObservedRunningTime="2026-11-03 21:41:54.318538365 +0000 UTC m=+723.197360649"
Nov 03 21:41:54.523003 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:54.522876    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:54.627598 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:54.627562    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:54.731208 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:54.731168    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:55.074848 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:55.074816    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:56.187889 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.187858    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"
Nov 03 21:41:56.663347 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.663308    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"]
Nov 03 21:41:56.672870 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.672841    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:56.687147 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.687117    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"]
Nov 03 21:41:56.696317 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.696289    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-05bc27-serving-cert\""
Nov 03 21:41:56.720708 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.720676    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-maas-api-tls\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:56.720881 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.720739    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-metrics-certs\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:56.720949 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.720916    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zllpj\" (UniqueName: \"kubernetes.io/projected/35187015-8165-4126-8832-adcad32ca770-kube-api-access-zllpj\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:56.721000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.720962    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35187015-8165-4126-8832-adcad32ca770-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:56.781854 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.781820    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:56.821690 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.821655    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-maas-api-tls\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:56.821872 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.821723    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-metrics-certs\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:56.821872 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:41:56.821812    4454 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-serving-cert: secret "maas-api-e2e-worker-w4-05bc27-serving-cert" not found
Nov 03 21:41:56.821872 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.821844    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zllpj\" (UniqueName: \"kubernetes.io/projected/35187015-8165-4126-8832-adcad32ca770-kube-api-access-zllpj\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:56.822045 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:41:56.821882    4454 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-maas-api-tls podName:35187015-8165-4126-8832-adcad32ca770 nodeName:}" failed. No retries permitted until 2026-11-03 21:41:57.321861015 +0000 UTC m=+726.200683290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-maas-api-tls") pod "maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm" (UID: "35187015-8165-4126-8832-adcad32ca770") : secret "maas-api-e2e-worker-w4-05bc27-serving-cert" not found
Nov 03 21:41:56.822045 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.821941    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35187015-8165-4126-8832-adcad32ca770-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:56.822516 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.822489    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35187015-8165-4126-8832-adcad32ca770-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:56.824165 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.824144    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-metrics-certs\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:56.842219 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.842191    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zllpj\" (UniqueName: \"kubernetes.io/projected/35187015-8165-4126-8832-adcad32ca770-kube-api-access-zllpj\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:56.898038 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.888096    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:56.982870 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:56.982778    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:57.325881 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:57.325806    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-maas-api-tls\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:57.328242 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:57.328223    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-maas-api-tls\") pod \"maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:57.588622 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:57.588537    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:57.599967 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:57.599921    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:57.696967 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:57.696932    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:57.899896 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:57.899861    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"]
Nov 03 21:41:57.903889 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:41:57.903864    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35187015_8165_4126_8832_adcad32ca770.slice/crio-da228def14aa69f097e2f905514f264cd6257f23738e1e1cc76870099aa44533 WatchSource:0}: Error finding container da228def14aa69f097e2f905514f264cd6257f23738e1e1cc76870099aa44533: Status 404 returned error can't find the container with id da228def14aa69f097e2f905514f264cd6257f23738e1e1cc76870099aa44533
Nov 03 21:41:58.204667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:58.204630    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:41:58.214410 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:58.204687    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:41:58.214410 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:58.206319    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"
Nov 03 21:41:58.257825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:58.257801    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:41:58.280448 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:58.280420    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm" event={"ID":"35187015-8165-4126-8832-adcad32ca770","Type":"ContainerStarted","Data":"da228def14aa69f097e2f905514f264cd6257f23738e1e1cc76870099aa44533"}
Nov 03 21:41:58.902596 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:58.902525    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:41:59.213029 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:59.212946    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"
Nov 03 21:41:59.287978 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:59.287945    4454 generic.go:356] "Generic (PLEG): container finished" podID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerID="ddecd189b98992a329b97f08595bf60f2970d964a525fb120b82ed9ce146b5e2" exitCode=0
Nov 03 21:41:59.288172 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:59.288012    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" event={"ID":"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515","Type":"ContainerDied","Data":"ddecd189b98992a329b97f08595bf60f2970d964a525fb120b82ed9ce146b5e2"}
Nov 03 21:41:59.288481 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:59.288459    4454 scope.go:122] "RemoveContainer" containerID="ddecd189b98992a329b97f08595bf60f2970d964a525fb120b82ed9ce146b5e2"
Nov 03 21:41:59.289996 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:59.289971    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm" event={"ID":"35187015-8165-4126-8832-adcad32ca770","Type":"ContainerStarted","Data":"f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef"}
Nov 03 21:41:59.290095 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:59.290032    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:41:59.290095 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:59.290058    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:41:59.290199 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:59.290140    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:41:59.290199 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:59.290190    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:41:59.362821 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:59.362761    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm" podStartSLOduration=2.998209129 podStartE2EDuration="3.362742157s" totalImagesPullingTime="364.533028ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:41:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:41:59.361484153 +0000 UTC m=+728.240306447" watchObservedRunningTime="2026-11-03 21:41:59.362742157 +0000 UTC m=+728.241564440"
Nov 03 21:41:59.421720 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:41:59.421688    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:42:00.287272 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:00.287224    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:42:00.297285 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:00.297252    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" event={"ID":"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515","Type":"ContainerStarted","Data":"dd7b09cf846413837cac64c2a533158cad3571614de2ffea80120e719b758bb4"}
Nov 03 21:42:00.655373 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:00.655339    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:42:00.655584 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:00.655478    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Nov 03 21:42:00.655584 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:00.655541    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Nov 03 21:42:01.057157 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:01.057052    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:01.240545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:01.240496    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"
Nov 03 21:42:01.655474 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:01.655440    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Nov 03 21:42:01.655897 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:01.655488    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Nov 03 21:42:02.247575 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:02.247546    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"
Nov 03 21:42:02.656536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:02.656291    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Nov 03 21:42:02.656536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:02.656367    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Nov 03 21:42:02.916593 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:02.916458    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:03.655390 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:03.655357    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Nov 03 21:42:03.655605 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:03.655411    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Nov 03 21:42:04.655215 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:04.654928    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Nov 03 21:42:04.655215 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:04.655023    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Nov 03 21:42:04.851243 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:04.851205    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"]
Nov 03 21:42:04.855203 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:04.855183    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:04.875621 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:04.875592    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-517149-serving-cert\""
Nov 03 21:42:04.895335 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:04.895293    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"]
Nov 03 21:42:04.949215 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:04.949123    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:04.952916 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:04.952880    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frk9w\" (UniqueName: \"kubernetes.io/projected/0a1497ee-7ded-402f-b081-9b8a3f7a8825-kube-api-access-frk9w\") pod \"maas-api-e2e-worker-w6-517149-645559575d-f5tl4\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:04.953149 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:04.953129    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1497ee-7ded-402f-b081-9b8a3f7a8825-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-517149-645559575d-f5tl4\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:04.953315 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:04.953298    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-maas-api-tls\") pod \"maas-api-e2e-worker-w6-517149-645559575d-f5tl4\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:04.953468 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:04.953453    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-metrics-certs\") pod \"maas-api-e2e-worker-w6-517149-645559575d-f5tl4\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:05.020956 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.020921    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:42:05.054954 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.054927    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frk9w\" (UniqueName: \"kubernetes.io/projected/0a1497ee-7ded-402f-b081-9b8a3f7a8825-kube-api-access-frk9w\") pod \"maas-api-e2e-worker-w6-517149-645559575d-f5tl4\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:05.055119 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.054963    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1497ee-7ded-402f-b081-9b8a3f7a8825-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-517149-645559575d-f5tl4\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:05.055119 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.054998    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-maas-api-tls\") pod \"maas-api-e2e-worker-w6-517149-645559575d-f5tl4\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:05.055119 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.055043    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-metrics-certs\") pod \"maas-api-e2e-worker-w6-517149-645559575d-f5tl4\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:05.056627 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.056600    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1497ee-7ded-402f-b081-9b8a3f7a8825-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-517149-645559575d-f5tl4\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:05.059531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.058142    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-maas-api-tls\") pod \"maas-api-e2e-worker-w6-517149-645559575d-f5tl4\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:05.059531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.058631    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-metrics-certs\") pod \"maas-api-e2e-worker-w6-517149-645559575d-f5tl4\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:05.086713 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.086679    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:05.094530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.091300    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frk9w\" (UniqueName: \"kubernetes.io/projected/0a1497ee-7ded-402f-b081-9b8a3f7a8825-kube-api-access-frk9w\") pod \"maas-api-e2e-worker-w6-517149-645559575d-f5tl4\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:05.181049 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.181011    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:05.197837 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.197804    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:05.306642 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.305587    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:42:05.595586 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.588720    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"]
Nov 03 21:42:05.832538 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.832351    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:42:05.833184 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.833150    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:42:05.834386 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:05.834367    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:42:06.362466 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:06.362421    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4" event={"ID":"0a1497ee-7ded-402f-b081-9b8a3f7a8825","Type":"ContainerStarted","Data":"9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c"}
Nov 03 21:42:06.362664 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:06.362492    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4" event={"ID":"0a1497ee-7ded-402f-b081-9b8a3f7a8825","Type":"ContainerStarted","Data":"07f066367b9840950ac15de5ce89c74815941a0810a758c465ba939e1fb91446"}
Nov 03 21:42:06.399262 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:06.399209    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4" podStartSLOduration=2.055940351 podStartE2EDuration="2.399196874s" totalImagesPullingTime="343.256523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:42:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:42:06.39833061 +0000 UTC m=+735.277152896" watchObservedRunningTime="2026-11-03 21:42:06.399196874 +0000 UTC m=+735.278019157"
Nov 03 21:42:07.330562 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:07.330496    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:07.368853 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:07.368819    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:07.774057 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:07.774022    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:08.398670 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:08.398633    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:08.517686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:08.517650    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:08.629566 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:08.628932    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:09.012016 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:09.011977    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:09.381241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:09.381209    4454 generic.go:356] "Generic (PLEG): container finished" podID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerID="021254eeddaafccc5fd66b10d434f48a69095d210d07373915b313da279876f2" exitCode=0
Nov 03 21:42:09.381429 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:09.381277    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" event={"ID":"b9a8e74a-6b9e-43df-b41e-20ecf8181814","Type":"ContainerDied","Data":"021254eeddaafccc5fd66b10d434f48a69095d210d07373915b313da279876f2"}
Nov 03 21:42:09.381704 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:09.381684    4454 scope.go:122] "RemoveContainer" containerID="021254eeddaafccc5fd66b10d434f48a69095d210d07373915b313da279876f2"
Nov 03 21:42:09.383038 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:09.383012    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:42:09.383125 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:09.383102    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:42:09.383166 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:09.383132    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:42:09.451147 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:09.451115    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:09.551259 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:09.551144    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:09.675263 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:09.675221    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:09.781049 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:09.781014    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:09.988987 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:09.988955    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:10.387421 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:10.387375    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:10.393381 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:10.393351    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" event={"ID":"b9a8e74a-6b9e-43df-b41e-20ecf8181814","Type":"ContainerStarted","Data":"b22f14209489eb3100c547e4ee76a85724471e18170cf26a73ccf16504afb7d0"}
Nov 03 21:42:11.039346 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:11.039314    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:42:11.040571 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:11.040549    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Nov 03 21:42:11.040698 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:11.040592    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Nov 03 21:42:11.882530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:11.882344    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:12.039394 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:12.039320    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Nov 03 21:42:12.039394 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:12.039375    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Nov 03 21:42:12.163972 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:12.163888    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:42:13.039402 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:13.039364    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Nov 03 21:42:13.039903 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:13.039427    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Nov 03 21:42:13.389856 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:13.389828    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:42:13.879538 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:13.879433    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:14.038712 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:14.038678    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Nov 03 21:42:14.038892 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:14.038735    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Nov 03 21:42:14.160819 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:14.160732    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:14.915184 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:14.915143    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:15.038710 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:15.038677    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Nov 03 21:42:15.038892 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:15.038728    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Nov 03 21:42:15.067536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:15.067065    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:16.042935 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:16.042907    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:42:16.043998 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:16.043982    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:42:16.258838 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:16.258802    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:17.796823 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:17.796787    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:42:18.212428 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:18.212397    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:18.976591 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:18.976550    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:19.287090 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:19.287001    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:19.487959 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:19.487906    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:19.921254 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:19.921218    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:20.024275 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:20.024236    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:20.129331 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:20.129277    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:20.419796 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:20.419759    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:21.388080 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:21.388041    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:22.385535 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:22.383010    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:22.391410 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:42:22.391376    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:42:24.259896 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:24.259860    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:26.277090 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:26.277042    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:27.579469 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:27.578309    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": read tcp 10.133.0.2:51348->10.133.0.38:15021: read: connection reset by peer" start-of-body=
Nov 03 21:42:27.579469 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:27.578500    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": read tcp 10.133.0.2:51348->10.133.0.38:15021: read: connection reset by peer"
Nov 03 21:42:28.048052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:28.048017    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:28.231571 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:28.231532    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:28.559822 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:28.559788    4454 generic.go:356] "Generic (PLEG): container finished" podID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerID="b55135cf5db3afe449733f517d589fd3a5612d789265301840d3e24202895fba" exitCode=0
Nov 03 21:42:28.560003 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:28.559859    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" event={"ID":"2588bcca-4842-455d-a4fd-7397fa0bdb5f","Type":"ContainerDied","Data":"b55135cf5db3afe449733f517d589fd3a5612d789265301840d3e24202895fba"}
Nov 03 21:42:28.560231 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:28.560218    4454 scope.go:122] "RemoveContainer" containerID="b55135cf5db3afe449733f517d589fd3a5612d789265301840d3e24202895fba"
Nov 03 21:42:28.561567 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:28.561544    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:42:28.561629 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:28.561612    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:42:28.561663 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:28.561643    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:42:28.644549 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:28.644522    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:28.947030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:28.946994    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:29.069530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:29.067120    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:29.190576 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:29.185033    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:29.589532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:29.587982    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" event={"ID":"2588bcca-4842-455d-a4fd-7397fa0bdb5f","Type":"ContainerStarted","Data":"d774c87dcd02ee26c5b237661769506c1351fda91c42e3863169616aab1bde70"}
Nov 03 21:42:29.710459 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:29.710425    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:29.787381 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:29.783929    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:42:29.787381 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:29.784049    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Nov 03 21:42:29.787381 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:29.784099    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Nov 03 21:42:29.815723 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:29.815681    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:29.925333 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:29.925300    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:30.276137 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:30.276040    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:30.785952 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:30.785906    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Nov 03 21:42:30.786376 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:30.785971    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Nov 03 21:42:30.959154 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:30.959111    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:31.397578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:31.397540    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:42:31.788948 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:31.788854    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Nov 03 21:42:31.789667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:31.789502    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Nov 03 21:42:31.999052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:31.999015    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:32.789246 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:32.789201    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Nov 03 21:42:32.790028 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:32.789986    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Nov 03 21:42:33.785862 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:33.785828    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Nov 03 21:42:33.786118 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:33.786095    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Nov 03 21:42:33.851715 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:33.851482    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:34.786853 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:34.786823    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:42:34.787150 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:34.787131    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:42:34.787846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:34.787822    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:42:35.557335 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:35.557291    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:36.054140 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:36.054107    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:36.411642 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:36.411592    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:37.355081 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:37.355045    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:39.140494 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:39.140459    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:40.856433 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:40.856330    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:41.948195 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:41.948154    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:42.656258 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:42.656221    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:43.895830 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.895799    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"]
Nov 03 21:42:43.900469 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.900438    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:43.925014 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.924983    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-4eced5\"/\"default-dockercfg-z2tkr\""
Nov 03 21:42:43.925175 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.924982    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-4eced5\"/\"premium-w1-4eced5-kserve-self-signed-certs\""
Nov 03 21:42:43.928121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.928098    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"]
Nov 03 21:42:43.945785 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.945756    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:43.956487 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.956440    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-4eced5\"/\"kube-root-ca.crt\""
Nov 03 21:42:43.956487 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.956444    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-4eced5\"/\"openshift-service-ca.crt\""
Nov 03 21:42:43.984178 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.984148    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-home\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:43.984178 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.984180    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-dshm\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:43.984363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.984344    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-model-cache\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:43.984397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.984379    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tls-certs\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:43.984492 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.984475    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tmp-dir\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:43.984555 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:43.984539    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x44r6\" (UniqueName: \"kubernetes.io/projected/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-kube-api-access-x44r6\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.091121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.085453    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-model-cache\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.091121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.085529    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tls-certs\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.091121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.085584    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tmp-dir\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.091121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.085621    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x44r6\" (UniqueName: \"kubernetes.io/projected/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-kube-api-access-x44r6\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.091121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.085687    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-home\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.091121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.085716    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-dshm\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.091121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.089556    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tmp-dir\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.091121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.089842    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-model-cache\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.091121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.090072    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-home\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.101121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.100083    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-dshm\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.104677 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.104650    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tls-certs\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.124295 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.123602    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x44r6\" (UniqueName: \"kubernetes.io/projected/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-kube-api-access-x44r6\") pod \"premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") " pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.215953 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.215874    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:44.458734 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.458699    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:44.525577 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.524436    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"]
Nov 03 21:42:44.547977 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:42:44.547595    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod300b5ac5_db5d_4a3a_b00d_0c8dc0a637ac.slice/crio-6b8ae227f700bb852292a572473b761fc6b870552f2ce56f3c1982e7c896d3df WatchSource:0}: Error finding container 6b8ae227f700bb852292a572473b761fc6b870552f2ce56f3c1982e7c896d3df: Status 404 returned error can't find the container with id 6b8ae227f700bb852292a572473b761fc6b870552f2ce56f3c1982e7c896d3df
Nov 03 21:42:44.673174 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.673136    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" event={"ID":"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac","Type":"ContainerStarted","Data":"5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1"}
Nov 03 21:42:44.673367 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.673186    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" event={"ID":"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac","Type":"ContainerStarted","Data":"6b8ae227f700bb852292a572473b761fc6b870552f2ce56f3c1982e7c896d3df"}
Nov 03 21:42:44.708180 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:44.708129    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podStartSLOduration=1.7081161630000001 podStartE2EDuration="1.708116163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:42:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:42:44.707922346 +0000 UTC m=+773.586744651" watchObservedRunningTime="2026-11-03 21:42:44.708116163 +0000 UTC m=+773.586938447"
Nov 03 21:42:45.216564 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:45.216530    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:45.910598 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:45.910553    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:42:46.082135 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:46.082094    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:47.859116 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:47.859067    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:48.661901 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:48.661866    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:49.636264 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:49.636230    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:51.442686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:51.442645    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:52.371046 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:42:52.371008    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:42:53.166960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:53.166921    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:54.216066 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.216028    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:54.227924 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.227901    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:54.448010 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.447953    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"]
Nov 03 21:42:54.453699 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.453294    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.466960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.466907    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-27fdf4-openshift-default-dockercfg-5cb9j\""
Nov 03 21:42:54.479924 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.479902    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"]
Nov 03 21:42:54.610892 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.610862    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-envoy\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.610892 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.610892    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-podinfo\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.611157 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.610914    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.611157 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.611047    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-data\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.611157 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.611152    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm4nz\" (UniqueName: \"kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-kube-api-access-bm4nz\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.611314 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.611176    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-credential-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.611314 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.611289    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c1f09c7-5b84-4789-9b2b-17657863cd18-istiod-ca-cert\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.611411 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.611327    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-token\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.611460 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.611418    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-certs\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.712799 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.712770    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm4nz\" (UniqueName: \"kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-kube-api-access-bm4nz\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.712975 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.712805    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-credential-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.712975 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.712866    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c1f09c7-5b84-4789-9b2b-17657863cd18-istiod-ca-cert\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.712975 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.712957    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-token\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.713140 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.713045    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-certs\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.713140 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.713096    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-envoy\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.713140 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.713131    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-podinfo\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.713281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.713165    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.713281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.713205    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-data\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.713374 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.713277    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-credential-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.713374 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.713342    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-certs\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.713553 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.713536    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.713628 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.713578    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c1f09c7-5b84-4789-9b2b-17657863cd18-istiod-ca-cert\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.713628 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.713615    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-data\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.715297 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.715276    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-envoy\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.715568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.715555    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-podinfo\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.730618 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.730566    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-token\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.730756 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.730738    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm4nz\" (UniqueName: \"kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-kube-api-access-bm4nz\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.752171 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.752148    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:42:54.767378 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.767353    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:54.850649 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:54.850613    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:55.091581 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:55.087435    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"]
Nov 03 21:42:55.139846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:55.139276    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:42:55.139846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:55.139366    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:42:55.139846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:55.139407    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:42:55.759001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:55.758651    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" event={"ID":"5c1f09c7-5b84-4789-9b2b-17657863cd18","Type":"ContainerStarted","Data":"cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e"}
Nov 03 21:42:55.759001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:55.758707    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" event={"ID":"5c1f09c7-5b84-4789-9b2b-17657863cd18","Type":"ContainerStarted","Data":"b22ca4aee32f465bbdaa6321b4e3729d00d4294b283b63aa1aa0f52f3f46b7d9"}
Nov 03 21:42:55.768027 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:55.767989    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:42:55.824325 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:55.824259    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" podStartSLOduration=1.824239041 podStartE2EDuration="1.824239041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:42:55.809980331 +0000 UTC m=+784.688802616" watchObservedRunningTime="2026-11-03 21:42:55.824239041 +0000 UTC m=+784.703061325"
Nov 03 21:42:56.648274 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:56.648235    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:56.777226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:56.777189    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 03 21:42:56.785778 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:56.785734    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" podUID="5c1f09c7-5b84-4789-9b2b-17657863cd18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 03 21:42:57.783539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:57.768863    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 03 21:42:57.783539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:57.768936    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" podUID="5c1f09c7-5b84-4789-9b2b-17657863cd18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 03 21:42:58.448995 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:58.448964    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:58.778683 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:58.778594    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 03 21:42:58.778920 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:58.778894    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" podUID="5c1f09c7-5b84-4789-9b2b-17657863cd18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 03 21:42:59.255686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:59.255647    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:42:59.767880 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:59.767843    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 03 21:42:59.768167 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:59.768138    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" podUID="5c1f09c7-5b84-4789-9b2b-17657863cd18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 03 21:42:59.987034 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:42:59.986996    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:43:00.234759 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:00.234717    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:00.781954 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:00.781907    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 03 21:43:00.782587 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:00.782550    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" podUID="5c1f09c7-5b84-4789-9b2b-17657863cd18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 03 21:43:01.833193 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:01.832789    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:43:01.833898 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:01.833872    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:43:01.834659 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:01.834635    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:43:02.046545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:02.046504    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:03.839390 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:03.839350    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:04.403589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.403557    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"]
Nov 03 21:43:04.408562 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.408543    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.433260 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.433230    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-517149-openshift-default-dockercfg-hr4lp\""
Nov 03 21:43:04.436388 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.436364    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"]
Nov 03 21:43:04.454720 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.454686    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.454874 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.454729    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-token\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.454874 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.454826    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7jxf\" (UniqueName: \"kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-kube-api-access-b7jxf\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.454972 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.454915    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-data\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.455003 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.454971    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istiod-ca-cert\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.455036 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.455004    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-certs\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.455165 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.455142    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-credential-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.455240 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.455229    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-envoy\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.455277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.455262    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-podinfo\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.556729 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.556688    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-data\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.556729 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.556725    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istiod-ca-cert\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.557028 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.556743    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-certs\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.557028 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.556792    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-credential-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.557028 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.556849    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-envoy\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.557028 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.556895    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-podinfo\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.557028 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.556938    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.557028 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.556962    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-token\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.557028 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.556988    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7jxf\" (UniqueName: \"kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-kube-api-access-b7jxf\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.557369 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.557194    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-data\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.557369 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.557285    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-credential-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.557682 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.557657    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.557777 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.557681    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-certs\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.557828 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.557771    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istiod-ca-cert\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.559322 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.559298    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-envoy\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.559573 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.559552    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-podinfo\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.574997 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.574970    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-token\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.575339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.575317    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7jxf\" (UniqueName: \"kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-kube-api-access-b7jxf\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:04.722285 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:04.722208    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:05.032349 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:05.032319    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"]
Nov 03 21:43:05.034769 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:43:05.034741    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32ce613f_1f5c_40f8_a895_ce9dbc0016e4.slice/crio-290a96be2fc8f7cc7cfe3cb3f60a2bcd9750084b7d50821338c9a8f121dd4dc5 WatchSource:0}: Error finding container 290a96be2fc8f7cc7cfe3cb3f60a2bcd9750084b7d50821338c9a8f121dd4dc5: Status 404 returned error can't find the container with id 290a96be2fc8f7cc7cfe3cb3f60a2bcd9750084b7d50821338c9a8f121dd4dc5
Nov 03 21:43:05.037062 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:05.037032    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:43:05.037123 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:05.037097    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:43:05.037160 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:05.037124    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:43:05.535003 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:05.534968    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:05.838207 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:05.838114    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7" event={"ID":"32ce613f-1f5c-40f8-a895-ce9dbc0016e4","Type":"ContainerStarted","Data":"221a3679e751c502fb01fce2e87429a8505cd5f36fa5cf835e1912469e8ff624"}
Nov 03 21:43:05.838207 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:05.838179    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7" event={"ID":"32ce613f-1f5c-40f8-a895-ce9dbc0016e4","Type":"ContainerStarted","Data":"290a96be2fc8f7cc7cfe3cb3f60a2bcd9750084b7d50821338c9a8f121dd4dc5"}
Nov 03 21:43:05.886706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:05.886653    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7" podStartSLOduration=1.886637906 podStartE2EDuration="1.886637906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:43:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:43:05.88620429 +0000 UTC m=+794.765026586" watchObservedRunningTime="2026-11-03 21:43:05.886637906 +0000 UTC m=+794.765460190"
Nov 03 21:43:06.722788 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:06.722751    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:06.724532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:06.724492    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 03 21:43:06.724598 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:06.724557    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7" podUID="32ce613f-1f5c-40f8-a895-ce9dbc0016e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 03 21:43:07.240635 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:07.240598    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:07.723741 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:07.723697    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 03 21:43:07.724141 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:07.723762    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7" podUID="32ce613f-1f5c-40f8-a895-ce9dbc0016e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 03 21:43:08.387342 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:08.387299    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:08.723144 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:08.723056    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 03 21:43:08.723144 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:08.723107    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7" podUID="32ce613f-1f5c-40f8-a895-ce9dbc0016e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 03 21:43:09.048210 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:09.048130    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:09.722775 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:09.722736    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 03 21:43:09.722947 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:09.722788    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7" podUID="32ce613f-1f5c-40f8-a895-ce9dbc0016e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 03 21:43:10.401990 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:10.401930    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:10.723239 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:10.723161    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Nov 03 21:43:10.723239 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:10.723221    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7" podUID="32ce613f-1f5c-40f8-a895-ce9dbc0016e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Nov 03 21:43:10.870015 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:10.869976    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:11.727361 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:11.727328    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:11.727751 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:11.727590    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:11.728314 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:11.728297    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:43:12.570447 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:12.570404    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:14.267131 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.267094    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:14.340341 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.340112    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:43:14.712461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.712424    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"]
Nov 03 21:43:14.717227 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.717195    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.736844 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.736809    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"]
Nov 03 21:43:14.745377 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.745348    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-e246f9\"/\"openshift-service-ca.crt\""
Nov 03 21:43:14.745569 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.745455    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-e246f9\"/\"premium-w0-e246f9-kserve-self-signed-certs\""
Nov 03 21:43:14.745569 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.745497    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-e246f9\"/\"kube-root-ca.crt\""
Nov 03 21:43:14.745569 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.745348    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-e246f9\"/\"default-dockercfg-pvxz5\""
Nov 03 21:43:14.779673 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.779628    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:14.790926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.790896    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tmp-dir\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.790926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.790929    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-dshm\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.791152 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.790954    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99cc6\" (UniqueName: \"kubernetes.io/projected/6450e3ec-6e14-4520-94e9-3d41d5b578f8-kube-api-access-99cc6\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.791152 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.791091    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-home\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.791152 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.791120    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-model-cache\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.791331 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.791214    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tls-certs\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.892952 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.892908    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-model-cache\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.892952 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.892945    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tls-certs\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.893228 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.893042    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tmp-dir\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.893228 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.893063    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-dshm\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.893228 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.893086    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99cc6\" (UniqueName: \"kubernetes.io/projected/6450e3ec-6e14-4520-94e9-3d41d5b578f8-kube-api-access-99cc6\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.893228 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.893124    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-home\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.893465 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.893376    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-model-cache\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.893465 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.893425    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tmp-dir\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.893586 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.893493    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-home\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.895589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.895565    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-dshm\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.895589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.895589    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tls-certs\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:14.910786 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:14.910763    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99cc6\" (UniqueName: \"kubernetes.io/projected/6450e3ec-6e14-4520-94e9-3d41d5b578f8-kube-api-access-99cc6\") pod \"premium-w0-e246f9-kserve-f5bf57d4b-c56kv\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") " pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:15.037622 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:15.036741    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:15.424440 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:15.424380    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"]
Nov 03 21:43:15.433794 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:43:15.433759    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6450e3ec_6e14_4520_94e9_3d41d5b578f8.slice/crio-4dbc8fc3fa60b0fccd604c398baca904276667de33b5f0015f4218da53ae7902 WatchSource:0}: Error finding container 4dbc8fc3fa60b0fccd604c398baca904276667de33b5f0015f4218da53ae7902: Status 404 returned error can't find the container with id 4dbc8fc3fa60b0fccd604c398baca904276667de33b5f0015f4218da53ae7902
Nov 03 21:43:15.896252 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:15.896196    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" event={"ID":"6450e3ec-6e14-4520-94e9-3d41d5b578f8","Type":"ContainerStarted","Data":"1639e42dd36b2016482a52500c1fa7d0255ecd3d2c76e2db3e04c830d61efbc5"}
Nov 03 21:43:15.896596 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:15.896569    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" event={"ID":"6450e3ec-6e14-4520-94e9-3d41d5b578f8","Type":"ContainerStarted","Data":"4dbc8fc3fa60b0fccd604c398baca904276667de33b5f0015f4218da53ae7902"}
Nov 03 21:43:15.934262 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:15.934201    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podStartSLOduration=1.934187157 podStartE2EDuration="1.934187157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:43:15.933723789 +0000 UTC m=+804.812546073" watchObservedRunningTime="2026-11-03 21:43:15.934187157 +0000 UTC m=+804.813009440"
Nov 03 21:43:16.038160 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:16.038127    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:21.174060 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:21.174020    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:21.261825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:21.261783    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:21.358603 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:21.358566    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:22.371213 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:43:22.371172    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:43:23.151952 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:23.151908    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:23.864458 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:23.864425    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:24.841208 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:24.841176    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:25.038229 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:25.038202    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:25.050210 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:25.050187    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:25.957557 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:25.957528    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:43:26.599468 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:26.599433    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:28.400994 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:28.400952    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:30.248082 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:30.248043    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:31.857776 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:31.857721    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:33.550750 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:33.550711    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:35.267276 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:35.267243    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:36.181940 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:36.181885    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:36.191655 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:36.191626    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:43:37.058751 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:37.058702    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:38.883146 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:38.883095    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:40.636355 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:40.636318    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:42.408166 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:42.408124    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:44.181426 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:44.181391    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:46.001677 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:46.001625    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:47.662004 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:47.661971    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:49.352409 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:49.352369    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:51.214707 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:51.214410    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:52.370309 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:43:52.370267    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:43:52.945359 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:52.945321    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:54.900364 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:54.900328    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:55.051772 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.051735    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"]
Nov 03 21:43:55.084535 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.075435    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.163305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.099017    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"]
Nov 03 21:43:55.163305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.107064    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-envoy\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.163305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.109671    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-certs\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.163305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.109700    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-data\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.163305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.109742    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-token\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.163305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.109851    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istiod-ca-cert\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.163305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.109887    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-credential-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.163305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.109971    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.163305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.110002    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85mh7\" (UniqueName: \"kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-kube-api-access-85mh7\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.163305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.110033    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-podinfo\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.188736 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.188706    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-05bc27-openshift-default-dockercfg-bxn7k\""
Nov 03 21:43:55.250565 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.250523    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istiod-ca-cert\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.251625 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.251599    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-credential-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.252173 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.252154    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.252635 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.252616    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85mh7\" (UniqueName: \"kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-kube-api-access-85mh7\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.253309 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.253288    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-podinfo\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.253663 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.253633    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-envoy\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.253757 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.253689    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-certs\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.253757 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.253714    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-data\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.253869 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.253753    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-token\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.254026 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.252066    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-credential-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.254207 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.252567    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.254207 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.251531    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istiod-ca-cert\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.254363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.254288    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-certs\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.254419 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.254369    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-data\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.256565 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.256540    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-envoy\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.256645 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.256569    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-podinfo\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.303264 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.303227    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85mh7\" (UniqueName: \"kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-kube-api-access-85mh7\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.328919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.322026    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-token\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.336881 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.336836    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"]
Nov 03 21:43:55.343269 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.343237    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.360077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.360056    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-dd5620-openshift-default-dockercfg-cm274\""
Nov 03 21:43:55.381540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.378365    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"]
Nov 03 21:43:55.457137 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.457054    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-envoy\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.457137 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.457097    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-socket\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.457347 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.457243    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxdp2\" (UniqueName: \"kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-kube-api-access-cxdp2\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.457347 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.457312    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-certs\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.457462 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.457364    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istiod-ca-cert\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.457522 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.457458    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-token\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.457587 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.457565    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-credential-socket\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.457636 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.457598    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-data\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.457690 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.457638    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-podinfo\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.549696 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.549652    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:55.562472 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.562368    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-credential-socket\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.562472 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.562411    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-data\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.562472 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.562453    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-podinfo\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.562769 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.562545    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-envoy\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.562769 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.562591    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-socket\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.562769 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.562690    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxdp2\" (UniqueName: \"kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-kube-api-access-cxdp2\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.562769 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.562764    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-certs\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.562966 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.562787    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istiod-ca-cert\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.562966 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.562833    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-token\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.564537 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.563618    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-socket\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.564537 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.563871    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-certs\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.564537 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.564233    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istiod-ca-cert\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.564537 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.564475    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-data\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.564784 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.564744    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-credential-socket\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.566400 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.566355    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-envoy\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.567335 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.567295    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-podinfo\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.582216 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.581458    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-token\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.582360 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.582323    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxdp2\" (UniqueName: \"kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-kube-api-access-cxdp2\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.660256 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.659896    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:55.890992 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.890955    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"]
Nov 03 21:43:55.896215 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:43:55.896186    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod397ba3eb_ea51_46a3_ad14_b52e3d107db1.slice/crio-ca11efb9c6452c59e5b007a3ae09d022656b5933853fceeb62c8947f8798597f WatchSource:0}: Error finding container ca11efb9c6452c59e5b007a3ae09d022656b5933853fceeb62c8947f8798597f: Status 404 returned error can't find the container with id ca11efb9c6452c59e5b007a3ae09d022656b5933853fceeb62c8947f8798597f
Nov 03 21:43:55.898705 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.898660    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:43:55.898819 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.898747    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:43:55.898819 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.898785    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:43:55.965639 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.965613    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"]
Nov 03 21:43:55.968845 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:43:55.968820    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6c694f6_eeeb_4b68_89ca_64447fa9955d.slice/crio-d7d6bab45a60f20bfafe0999265c4b4a9946aeb6f64ee49bfeef266f7a92b15b WatchSource:0}: Error finding container d7d6bab45a60f20bfafe0999265c4b4a9946aeb6f64ee49bfeef266f7a92b15b: Status 404 returned error can't find the container with id d7d6bab45a60f20bfafe0999265c4b4a9946aeb6f64ee49bfeef266f7a92b15b
Nov 03 21:43:55.971650 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.971617    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:43:55.971723 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.971684    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:43:55.971723 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:55.971712    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:43:56.196588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:56.196550    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w" event={"ID":"c6c694f6-eeeb-4b68-89ca-64447fa9955d","Type":"ContainerStarted","Data":"10410980cf7cf028a2d1c434ac95d7b64807b6c85ac4088d50f7c464aa02cbd4"}
Nov 03 21:43:56.196790 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:56.196612    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w" event={"ID":"c6c694f6-eeeb-4b68-89ca-64447fa9955d","Type":"ContainerStarted","Data":"d7d6bab45a60f20bfafe0999265c4b4a9946aeb6f64ee49bfeef266f7a92b15b"}
Nov 03 21:43:56.198271 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:56.198239    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" event={"ID":"397ba3eb-ea51-46a3-ad14-b52e3d107db1","Type":"ContainerStarted","Data":"f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2"}
Nov 03 21:43:56.198387 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:56.198288    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" event={"ID":"397ba3eb-ea51-46a3-ad14-b52e3d107db1","Type":"ContainerStarted","Data":"ca11efb9c6452c59e5b007a3ae09d022656b5933853fceeb62c8947f8798597f"}
Nov 03 21:43:56.236624 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:56.236012    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w" podStartSLOduration=1.2359942959999999 podStartE2EDuration="1.235994296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:43:56.234027262 +0000 UTC m=+845.112849612" watchObservedRunningTime="2026-11-03 21:43:56.235994296 +0000 UTC m=+845.114816581"
Nov 03 21:43:56.275211 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:56.274555    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" podStartSLOduration=2.274495124 podStartE2EDuration="2.274495124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:43:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:43:56.272322164 +0000 UTC m=+845.151144464" watchObservedRunningTime="2026-11-03 21:43:56.274495124 +0000 UTC m=+845.153317412"
Nov 03 21:43:56.550160 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:56.550088    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:43:56.551268 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:56.551242    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 03 21:43:56.551323 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:56.551296    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" podUID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 03 21:43:56.672539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:56.669659    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:43:56.743231 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:56.743201    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:57.556579 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:57.556537    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 03 21:43:57.557061 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:57.556623    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" podUID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 03 21:43:57.669642 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:57.667142    4454 patch_prober.go:29] interesting pod/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 03 21:43:57.669642 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:57.667200    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w" podUID="c6c694f6-eeeb-4b68-89ca-64447fa9955d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 03 21:43:58.550097 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:58.550061    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 03 21:43:58.550292 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:58.550116    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" podUID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 03 21:43:58.669545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:58.669486    4454 patch_prober.go:29] interesting pod/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 03 21:43:58.670041 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:58.669568    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w" podUID="c6c694f6-eeeb-4b68-89ca-64447fa9955d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 03 21:43:58.750664 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:58.750622    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:43:59.550894 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:59.550859    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 03 21:43:59.551137 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:59.551115    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" podUID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 03 21:43:59.683533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:59.674186    4454 patch_prober.go:29] interesting pod/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 03 21:43:59.683533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:43:59.674247    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w" podUID="c6c694f6-eeeb-4b68-89ca-64447fa9955d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 03 21:44:00.551070 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:00.551031    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 03 21:44:00.551484 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:00.551451    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" podUID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 03 21:44:00.620294 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:00.620263    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:00.695545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:00.676934    4454 patch_prober.go:29] interesting pod/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 03 21:44:00.695545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:00.677009    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w" podUID="c6c694f6-eeeb-4b68-89ca-64447fa9955d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 03 21:44:01.550228 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:01.550193    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Nov 03 21:44:01.550404 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:01.550240    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" podUID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Nov 03 21:44:01.667534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:01.663847    4454 patch_prober.go:29] interesting pod/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 03 21:44:01.667534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:01.663909    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w" podUID="c6c694f6-eeeb-4b68-89ca-64447fa9955d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 03 21:44:02.454628 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:02.454589    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:02.643378 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:02.642282    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:44:02.643965 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:02.643941    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:44:02.645536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:02.645524    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:44:02.688551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:02.688414    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:44:02.688808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:02.688788    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:44:02.692667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:02.692643    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:44:04.259359 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:04.259275    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:06.044503 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:06.044467    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:07.761529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:07.761066    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:09.449644 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:09.449596    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:10.471529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:10.471479    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:44:11.163750 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:11.163709    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:13.029452 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:13.029405    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:14.756178 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:14.756139    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:16.416488 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:16.416357    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:18.166275 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:18.166239    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:19.960248 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:19.960211    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:21.882649 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:21.882613    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:22.381882 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:44:22.381842    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:44:23.563846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:23.563800    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:23.683149 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:23.683108    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:25.485944 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:25.485901    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:27.852672 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:27.852629    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:29.011540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:29.011120    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:29.120532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:29.120223    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:30.089754 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:30.088568    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:30.196125 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:30.196083    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:30.565747 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:30.565709    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:32.334837 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:32.334804    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:34.046733 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:34.046704    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:35.762307 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:35.762265    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:36.322501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:36.322461    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:36.504968 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:36.504933    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:37.646463 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:37.646427    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:37.907894 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:37.907805    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:38.015537 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:38.014989    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:39.358915 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:39.358879    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:41.146637 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:41.146594    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:42.036015 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:42.035984    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:44:42.997704 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:42.997673    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:45.872162 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:45.872122    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:47.855851 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:47.855809    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:49.660696 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:49.660655    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:50.787242 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.787207    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"]
Nov 03 21:44:50.790673 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.790656    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:50.815619 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.815587    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-0a6da1\"/\"openshift-service-ca.crt\""
Nov 03 21:44:50.815797 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.815630    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-0a6da1\"/\"default-dockercfg-rdxmf\""
Nov 03 21:44:50.815797 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.815735    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-0a6da1\"/\"kube-root-ca.crt\""
Nov 03 21:44:50.818044 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.818019    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"]
Nov 03 21:44:50.818497 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.818481    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-0a6da1\"/\"distinct-w5-0a6da1-kserve-self-signed-certs\""
Nov 03 21:44:50.835559 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.835523    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:50.901311 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.901277    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9fmw\" (UniqueName: \"kubernetes.io/projected/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-kube-api-access-q9fmw\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:50.901489 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.901336    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tls-certs\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:50.901570 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.901495    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tmp-dir\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:50.901570 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.901542    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-model-cache\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:50.901636 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.901620    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-home\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:50.901669 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:50.901658    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-dshm\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.009580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.009498    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9fmw\" (UniqueName: \"kubernetes.io/projected/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-kube-api-access-q9fmw\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.010172 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.010143    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tls-certs\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.010308 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.010255    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tmp-dir\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.010308 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.010286    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-model-cache\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.010415 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.010344    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-home\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.010415 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.010402    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-dshm\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.011028 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.011004    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-model-cache\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.011215 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.011075    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tmp-dir\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.011363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.011093    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-home\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.014110 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.014088    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-dshm\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.014382 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.014362    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tls-certs\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.039848 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.036863    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9fmw\" (UniqueName: \"kubernetes.io/projected/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-kube-api-access-q9fmw\") pod \"distinct-w5-0a6da1-kserve-7458dc659f-x6mfj\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") " pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.102741 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.102704    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:51.303168 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.303081    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:44:51.303842 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.303820    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:44:51.361550 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.361499    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:51.405168 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.405141    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"]
Nov 03 21:44:51.408395 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:44:51.408370    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2007d8ff_83cc_42bc_9f9d_766b4ffda89b.slice/crio-5f4023e8e56578f688c23124d17ce1b23487d79e978c40f02a2e556abaa56dd8 WatchSource:0}: Error finding container 5f4023e8e56578f688c23124d17ce1b23487d79e978c40f02a2e556abaa56dd8: Status 404 returned error can't find the container with id 5f4023e8e56578f688c23124d17ce1b23487d79e978c40f02a2e556abaa56dd8
Nov 03 21:44:51.695568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.695528    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" event={"ID":"2007d8ff-83cc-42bc-9f9d-766b4ffda89b","Type":"ContainerStarted","Data":"37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2"}
Nov 03 21:44:51.695771 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.695575    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" event={"ID":"2007d8ff-83cc-42bc-9f9d-766b4ffda89b","Type":"ContainerStarted","Data":"5f4023e8e56578f688c23124d17ce1b23487d79e978c40f02a2e556abaa56dd8"}
Nov 03 21:44:51.733594 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:51.733517    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podStartSLOduration=1.733490677 podStartE2EDuration="1.733490677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:44:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:44:51.73124548 +0000 UTC m=+900.610067763" watchObservedRunningTime="2026-11-03 21:44:51.733490677 +0000 UTC m=+900.612312959"
Nov 03 21:44:52.103476 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:52.103435    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:44:52.371180 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:44:52.371101    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:44:53.143131 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:53.143099    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:54.778779 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:54.778731    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:56.450749 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:56.450704    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:44:58.234963 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:44:58.234920    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:00.188534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:00.182609    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:01.061992 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:01.061955    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:01.103341 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:01.103311    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:45:01.115684 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:01.115656    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:45:01.757305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:01.757273    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:45:01.977178 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:01.977144    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:03.759670 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:03.759636    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:04.718708 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.718668    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"]
Nov 03 21:45:04.722852 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.722830    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.745990 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.745953    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"]
Nov 03 21:45:04.806951 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.806917    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhlvz\" (UniqueName: \"kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-kube-api-access-mhlvz\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.806951 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.806954    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-credential-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.807461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.807090    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.807461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.807130    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istiod-ca-cert\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.807461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.807214    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-certs\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.807461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.807292    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-envoy\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.807461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.807384    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-podinfo\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.807461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.807452    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-token\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.807711 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.807490    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-data\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.908692 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.908661    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-podinfo\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.908869 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.908716    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-token\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.908869 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.908749    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-data\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.908869 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.908771    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhlvz\" (UniqueName: \"kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-kube-api-access-mhlvz\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.908869 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.908789    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-credential-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.908869 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.908860    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.909131 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.908889    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istiod-ca-cert\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.909131 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.909009    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-certs\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.909131 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.909077    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-envoy\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.909272 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.909161    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-data\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.909272 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.909227    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-credential-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.909272 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.909244    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.909422 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.909353    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-certs\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.909733 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.909711    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istiod-ca-cert\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.911187 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.911167    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-envoy\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.911281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.911175    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-podinfo\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.936247 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.936214    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhlvz\" (UniqueName: \"kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-kube-api-access-mhlvz\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:04.936368 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:04.936216    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-token\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:05.036118 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.036047    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:05.110634 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.110604    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"]
Nov 03 21:45:05.115615 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.115592    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.129242 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.129203    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-e246f9\"/\"distinct-w0-e246f9-kserve-self-signed-certs\""
Nov 03 21:45:05.129793 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.129775    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"]
Nov 03 21:45:05.148602 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.148533    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:05.212641 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.212610    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-dshm\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.212797 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.212719    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tmp-dir\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.212797 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.212748    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-model-cache\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.212874 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.212855    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-home\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.212919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.212907    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tls-certs\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.212967 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.212955    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2s2j\" (UniqueName: \"kubernetes.io/projected/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-kube-api-access-s2s2j\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.321580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.320215    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tmp-dir\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.321580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.320287    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-model-cache\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.321580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.320393    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-home\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.321580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.320456    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tls-certs\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.321580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.320562    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2s2j\" (UniqueName: \"kubernetes.io/projected/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-kube-api-access-s2s2j\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.321580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.320644    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-dshm\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.321580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.321539    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tmp-dir\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.326581 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.326555    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-model-cache\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.330427 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.330141    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-home\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.345783 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.345741    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2s2j\" (UniqueName: \"kubernetes.io/projected/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-kube-api-access-s2s2j\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.345783 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.345754    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-dshm\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.352259 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.352160    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tls-certs\") pod \"distinct-w0-e246f9-kserve-5d9689d6c7-vj57q\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") " pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.391276 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.391243    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"]
Nov 03 21:45:05.394196 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:45:05.394170    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd15fe5f_6e59_4763_af37_ddd5493e1af8.slice/crio-73595dff7289f117d245ba4fdaa8912053bc6e761a65d2ae91d9d399f67d3106 WatchSource:0}: Error finding container 73595dff7289f117d245ba4fdaa8912053bc6e761a65d2ae91d9d399f67d3106: Status 404 returned error can't find the container with id 73595dff7289f117d245ba4fdaa8912053bc6e761a65d2ae91d9d399f67d3106
Nov 03 21:45:05.396586 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.396553    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:45:05.396671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.396617    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:45:05.396671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.396645    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:45:05.431096 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.431072    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:05.578752 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.578654    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:05.735390 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.735363    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"]
Nov 03 21:45:05.739688 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:45:05.739662    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf09c8d95_070e_4706_8ddd_92ddaea9c8d8.slice/crio-b50e8e5073416646a53d35a95ef98b08ef2f08954464d7b0f38b466755544885 WatchSource:0}: Error finding container b50e8e5073416646a53d35a95ef98b08ef2f08954464d7b0f38b466755544885: Status 404 returned error can't find the container with id b50e8e5073416646a53d35a95ef98b08ef2f08954464d7b0f38b466755544885
Nov 03 21:45:05.774154 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.774117    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" event={"ID":"f09c8d95-070e-4706-8ddd-92ddaea9c8d8","Type":"ContainerStarted","Data":"b50e8e5073416646a53d35a95ef98b08ef2f08954464d7b0f38b466755544885"}
Nov 03 21:45:05.776319 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.776291    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk" event={"ID":"bd15fe5f-6e59-4763-af37-ddd5493e1af8","Type":"ContainerStarted","Data":"5d165d969de9a28eedd053969446c78aca3fb88e4f9db36c9bf926b779c8ffa6"}
Nov 03 21:45:05.776412 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.776343    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk" event={"ID":"bd15fe5f-6e59-4763-af37-ddd5493e1af8","Type":"ContainerStarted","Data":"73595dff7289f117d245ba4fdaa8912053bc6e761a65d2ae91d9d399f67d3106"}
Nov 03 21:45:05.818713 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:05.818654    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk" podStartSLOduration=1.818635058 podStartE2EDuration="1.818635058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:05.814811038 +0000 UTC m=+914.693633322" watchObservedRunningTime="2026-11-03 21:45:05.818635058 +0000 UTC m=+914.697457344"
Nov 03 21:45:06.037158 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:06.037112    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:06.038831 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:06.038798    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 21:45:06.038957 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:06.038864    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk" podUID="bd15fe5f-6e59-4763-af37-ddd5493e1af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 21:45:06.789479 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:06.789440    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" event={"ID":"f09c8d95-070e-4706-8ddd-92ddaea9c8d8","Type":"ContainerStarted","Data":"59f0e5bdf5aff05f6e56cdf7f83355d16c440e5aabc8b7ccf76465d5c1ef7503"}
Nov 03 21:45:06.828337 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:06.828290    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podStartSLOduration=1.8282764070000002 podStartE2EDuration="1.828276407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:06.826337056 +0000 UTC m=+915.705159339" watchObservedRunningTime="2026-11-03 21:45:06.828276407 +0000 UTC m=+915.707098740"
Nov 03 21:45:07.037034 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:07.037002    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 21:45:07.037194 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:07.037059    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk" podUID="bd15fe5f-6e59-4763-af37-ddd5493e1af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 21:45:07.398785 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:07.398729    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:07.431670 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:07.431640    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:08.036583 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.036549    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 21:45:08.037013 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.036612    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk" podUID="bd15fe5f-6e59-4763-af37-ddd5493e1af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 21:45:08.136210 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.136174    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"]
Nov 03 21:45:08.140655 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.140630    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.184873 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.184838    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-517149\"/\"default-dockercfg-jbw89\""
Nov 03 21:45:08.193578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.185191    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-517149\"/\"openshift-service-ca.crt\""
Nov 03 21:45:08.193796 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.185272    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-517149\"/\"kube-root-ca.crt\""
Nov 03 21:45:08.193873 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.185342    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-517149\"/\"distinct-w6-517149-kserve-self-signed-certs\""
Nov 03 21:45:08.193873 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.193152    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:08.195561 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.195536    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"]
Nov 03 21:45:08.293535 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.293418    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c4344c2-f323-4ed4-b879-faf1437d6f58-tls-certs\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.293687 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.293592    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-home\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.293724 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.293685    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-dshm\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.293758 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.293723    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-tmp-dir\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.293866 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.293852    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-model-cache\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.293901 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.293874    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhwlz\" (UniqueName: \"kubernetes.io/projected/6c4344c2-f323-4ed4-b879-faf1437d6f58-kube-api-access-xhwlz\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.346785 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.346743    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:45:08.401326 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.401285    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c4344c2-f323-4ed4-b879-faf1437d6f58-tls-certs\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.402134 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.402107    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-home\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.422725 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.422686    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-dshm\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.422920 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.422757    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-tmp-dir\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.422920 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.422876    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-model-cache\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.422920 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.422900    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhwlz\" (UniqueName: \"kubernetes.io/projected/6c4344c2-f323-4ed4-b879-faf1437d6f58-kube-api-access-xhwlz\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.423293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.402871    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-home\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.423846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.423822    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-tmp-dir\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.423931 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.423833    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-model-cache\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.429092 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.428149    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-dshm\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.431846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.431821    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c4344c2-f323-4ed4-b879-faf1437d6f58-tls-certs\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.441207 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.441186    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhwlz\" (UniqueName: \"kubernetes.io/projected/6c4344c2-f323-4ed4-b879-faf1437d6f58-kube-api-access-xhwlz\") pod \"distinct-w6-517149-kserve-6f88746b98-gv2zm\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") " pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.460119 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.459790    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:08.790071 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.790042    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"]
Nov 03 21:45:08.791918 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:45:08.791888    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c4344c2_f323_4ed4_b879_faf1437d6f58.slice/crio-c85304f2a6cc46e83f82e7e9754dcd2f78ee8133c12974fe803624e07489499b WatchSource:0}: Error finding container c85304f2a6cc46e83f82e7e9754dcd2f78ee8133c12974fe803624e07489499b: Status 404 returned error can't find the container with id c85304f2a6cc46e83f82e7e9754dcd2f78ee8133c12974fe803624e07489499b
Nov 03 21:45:08.800795 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:08.800767    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" event={"ID":"6c4344c2-f323-4ed4-b879-faf1437d6f58","Type":"ContainerStarted","Data":"c85304f2a6cc46e83f82e7e9754dcd2f78ee8133c12974fe803624e07489499b"}
Nov 03 21:45:09.037003 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:09.036963    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 21:45:09.037408 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:09.037036    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk" podUID="bd15fe5f-6e59-4763-af37-ddd5493e1af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 21:45:09.293527 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:09.290017    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:09.827130 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:09.827092    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" event={"ID":"6c4344c2-f323-4ed4-b879-faf1437d6f58","Type":"ContainerStarted","Data":"8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158"}
Nov 03 21:45:09.920814 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:09.920736    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podStartSLOduration=1.9207151740000001 podStartE2EDuration="1.920715174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:09.915437526 +0000 UTC m=+918.794259816" watchObservedRunningTime="2026-11-03 21:45:09.920715174 +0000 UTC m=+918.799537459"
Nov 03 21:45:09.953242 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:09.953198    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"]
Nov 03 21:45:09.960316 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:09.960136    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.032532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.024637    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"]
Nov 03 21:45:10.096109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.078409    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-certs\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.096109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.078474    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istiod-ca-cert\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.096109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.078544    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-data\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.096109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.078574    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-envoy\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.096109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.078605    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-token\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.096109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.078656    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-credential-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.096109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.078698    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p7vw\" (UniqueName: \"kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-kube-api-access-9p7vw\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.096109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.078761    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.096109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.078818    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 21:45:10.096109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.078827    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-podinfo\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.096109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.078857    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk" podUID="bd15fe5f-6e59-4763-af37-ddd5493e1af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 21:45:10.183604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.181234    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istiod-ca-cert\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.183604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.181307    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-data\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.183604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.181341    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-envoy\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.183604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.181368    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-token\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.183604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.181414    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-credential-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.183604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.181454    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p7vw\" (UniqueName: \"kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-kube-api-access-9p7vw\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.183604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.181563    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.183604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.181638    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-podinfo\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.183604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.181759    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-certs\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.183604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.182328    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-certs\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.183604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.182989    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istiod-ca-cert\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.183604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.183239    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-data\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.214026 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.187101    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.214026 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.195159    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-credential-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.235878 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.235807    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p7vw\" (UniqueName: \"kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-kube-api-access-9p7vw\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.252540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.252487    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-envoy\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.253388 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.253337    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-podinfo\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.254487 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.254360    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-token\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.468681 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.468592    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:10.496948 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.496917    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:10.812763 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.812690    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"]
Nov 03 21:45:10.816807 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:45:10.816780    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d31fd0d_3343_45cb_aa88_297d86d9ddfb.slice/crio-4fcb75d5e9535288e70357d72be16732076b0037952d56a042c0611a88274a0d WatchSource:0}: Error finding container 4fcb75d5e9535288e70357d72be16732076b0037952d56a042c0611a88274a0d: Status 404 returned error can't find the container with id 4fcb75d5e9535288e70357d72be16732076b0037952d56a042c0611a88274a0d
Nov 03 21:45:10.819384 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.819353    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:45:10.819440 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.819424    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:45:10.819476 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.819453    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:45:10.849157 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:10.849121    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz" event={"ID":"3d31fd0d-3343-45cb-aa88-297d86d9ddfb","Type":"ContainerStarted","Data":"4fcb75d5e9535288e70357d72be16732076b0037952d56a042c0611a88274a0d"}
Nov 03 21:45:11.036887 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:11.036852    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 21:45:11.037044 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:11.036906    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk" podUID="bd15fe5f-6e59-4763-af37-ddd5493e1af8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 21:45:11.384108 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:11.384073    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:11.855870 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:11.855828    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz" event={"ID":"3d31fd0d-3343-45cb-aa88-297d86d9ddfb","Type":"ContainerStarted","Data":"e959e612e10ffeec2370d5861c1221a35af42015f4dd802e6a9fcd706619e5a4"}
Nov 03 21:45:12.251522 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:12.251430    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:12.251761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:12.251741    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:12.252618 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:12.252597    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:45:12.304832 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:12.304782    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz" podStartSLOduration=3.304766223 podStartE2EDuration="3.304766223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:11.929290827 +0000 UTC m=+920.808113110" watchObservedRunningTime="2026-11-03 21:45:12.304766223 +0000 UTC m=+921.183588507"
Nov 03 21:45:12.497240 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:12.497204    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:12.499123 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:12.499096    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:45:12.499246 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:12.499143    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz" podUID="3d31fd0d-3343-45cb-aa88-297d86d9ddfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:45:13.289829 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.289789    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"]
Nov 03 21:45:13.294699 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.294677    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.325206 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.325183    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-4eced5\"/\"distinct-w1-4eced5-kserve-self-signed-certs\""
Nov 03 21:45:13.340795 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.340770    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"]
Nov 03 21:45:13.389998 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.389368    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-model-cache\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.389998 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.389459    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-dshm\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.389998 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.389529    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-home\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.389998 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.389561    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-tmp-dir\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.389998 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.389601    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmbjw\" (UniqueName: \"kubernetes.io/projected/a2700b70-1f64-4db1-b6c2-552669059d93-kube-api-access-hmbjw\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.389998 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.389643    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2700b70-1f64-4db1-b6c2-552669059d93-tls-certs\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.405835 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.405801    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:13.492153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.492110    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmbjw\" (UniqueName: \"kubernetes.io/projected/a2700b70-1f64-4db1-b6c2-552669059d93-kube-api-access-hmbjw\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.492343 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.492170    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2700b70-1f64-4db1-b6c2-552669059d93-tls-certs\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.492343 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.492305    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-model-cache\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.492459 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.492370    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-dshm\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.492459 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.492428    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-home\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.492606 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.492464    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-tmp-dir\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.493022 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.492961    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-tmp-dir\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.493301 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.493259    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-home\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.493301 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.493285    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-model-cache\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.495282 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.495259    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-dshm\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.495443 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.495424    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2700b70-1f64-4db1-b6c2-552669059d93-tls-certs\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.497493 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.497464    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:45:13.497946 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.497567    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz" podUID="3d31fd0d-3343-45cb-aa88-297d86d9ddfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:45:13.512582 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.512558    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmbjw\" (UniqueName: \"kubernetes.io/projected/a2700b70-1f64-4db1-b6c2-552669059d93-kube-api-access-hmbjw\") pod \"distinct-w1-4eced5-kserve-66d4d469fb-vxfbd\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:13.625614 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:13.622583    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:14.028861 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:14.028789    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"]
Nov 03 21:45:14.032071 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:45:14.032046    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2700b70_1f64_4db1_b6c2_552669059d93.slice/crio-2143d679ed21daae8892dfb3fa9d433004cb3e51c029875352226873418de37d WatchSource:0}: Error finding container 2143d679ed21daae8892dfb3fa9d433004cb3e51c029875352226873418de37d: Status 404 returned error can't find the container with id 2143d679ed21daae8892dfb3fa9d433004cb3e51c029875352226873418de37d
Nov 03 21:45:14.143032 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:14.142993    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:14.497595 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:14.497553    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:45:14.497769 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:14.497619    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz" podUID="3d31fd0d-3343-45cb-aa88-297d86d9ddfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:45:14.897254 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:14.897215    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" event={"ID":"a2700b70-1f64-4db1-b6c2-552669059d93","Type":"ContainerStarted","Data":"bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128"}
Nov 03 21:45:14.897254 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:14.897267    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" event={"ID":"a2700b70-1f64-4db1-b6c2-552669059d93","Type":"ContainerStarted","Data":"2143d679ed21daae8892dfb3fa9d433004cb3e51c029875352226873418de37d"}
Nov 03 21:45:14.935079 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:14.935028    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podStartSLOduration=1.935015223 podStartE2EDuration="1.935015223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:14.931812988 +0000 UTC m=+923.810635290" watchObservedRunningTime="2026-11-03 21:45:14.935015223 +0000 UTC m=+923.813837507"
Nov 03 21:45:15.431806 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:15.431771    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:15.444430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:15.444408    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:15.497410 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:15.497379    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:45:15.497622 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:15.497431    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz" podUID="3d31fd0d-3343-45cb-aa88-297d86d9ddfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:45:15.624991 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:15.624966    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:15.913439 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:15.913413    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:45:16.497988 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:16.497954    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:45:16.498173 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:16.498008    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz" podUID="3d31fd0d-3343-45cb-aa88-297d86d9ddfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:45:16.830116 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:16.830030    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:17.732196 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:17.732140    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:17.733083 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:17.733052    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:17.734586 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:17.734563    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:45:18.466122 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:18.466089    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:18.479700 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:18.479676    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:18.979289 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:18.979248    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:45:19.180969 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:19.180931    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:21.074458 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:21.074422    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:21.325627 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:21.325533    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:22.371777 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:45:22.371742    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:45:22.873853 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:22.873816    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:23.624884 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:23.624853    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:23.636993 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:23.636965    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:24.005593 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:24.005488    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:45:24.577782 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:24.577749    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:26.369840 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:26.369807    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:28.242838 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:28.242799    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:29.996585 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:29.996547    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:32.129554 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:32.129370    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:35.229946 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:35.229866    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:36.402120 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:36.402084    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:37.050016 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:37.049981    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:38.751645 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:38.751605    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:40.470328 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:40.470291    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:42.161851 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:42.161816    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:43.095282 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.095246    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"]
Nov 03 21:45:43.100089 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.100065    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.126494 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.126465    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-05bc27\"/\"kube-root-ca.crt\""
Nov 03 21:45:43.126695 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.126610    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-05bc27\"/\"default-dockercfg-dc6k9\""
Nov 03 21:45:43.126830 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.126637    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-05bc27\"/\"distinct-w4-05bc27-kserve-self-signed-certs\""
Nov 03 21:45:43.126907 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.126694    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-05bc27\"/\"openshift-service-ca.crt\""
Nov 03 21:45:43.130102 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.130076    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"]
Nov 03 21:45:43.143248 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.143221    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-home\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.143413 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.143284    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxskp\" (UniqueName: \"kubernetes.io/projected/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-kube-api-access-bxskp\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.143413 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.143404    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-dshm\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.143601 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.143445    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-model-cache\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.143601 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.143547    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tmp-dir\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.143806 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.143780    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tls-certs\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.149223 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.149198    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:43.245140 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.245110    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-home\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.245692 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.245156    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxskp\" (UniqueName: \"kubernetes.io/projected/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-kube-api-access-bxskp\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.245692 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.245190    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-dshm\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.245692 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.245213    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-model-cache\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.245692 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.245238    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tmp-dir\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.245692 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.245324    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tls-certs\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.245692 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.245599    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-model-cache\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.245692 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.245621    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-home\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.245984 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.245731    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tmp-dir\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.247625 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.247605    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-dshm\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.247788 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.247771    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tls-certs\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.262706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.262684    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxskp\" (UniqueName: \"kubernetes.io/projected/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-kube-api-access-bxskp\") pod \"distinct-w4-05bc27-kserve-79f7ff8454-k4r82\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") " pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.413129 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.413044    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:43.728459 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.728381    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"]
Nov 03 21:45:43.731626 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:45:43.731599    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b4acf6f_77ba_468e_800e_0cf7e8a8a9b9.slice/crio-aeb35e72dc374ae7e0e4c20cbc915d6f5990d49852c98583596a9e4af2038465 WatchSource:0}: Error finding container aeb35e72dc374ae7e0e4c20cbc915d6f5990d49852c98583596a9e4af2038465: Status 404 returned error can't find the container with id aeb35e72dc374ae7e0e4c20cbc915d6f5990d49852c98583596a9e4af2038465
Nov 03 21:45:43.896415 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:43.896376    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:44.303606 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:44.303568    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" event={"ID":"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9","Type":"ContainerStarted","Data":"79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6"}
Nov 03 21:45:44.303989 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:44.303619    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" event={"ID":"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9","Type":"ContainerStarted","Data":"aeb35e72dc374ae7e0e4c20cbc915d6f5990d49852c98583596a9e4af2038465"}
Nov 03 21:45:44.341416 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:44.341345    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podStartSLOduration=1.341329693 podStartE2EDuration="1.341329693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:44.339805282 +0000 UTC m=+953.218627568" watchObservedRunningTime="2026-11-03 21:45:44.341329693 +0000 UTC m=+953.220151975"
Nov 03 21:45:44.413201 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:44.413171    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:45.649926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:45.649888    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:47.377753 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:47.377716    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:48.952135 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:48.952097    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:50.663374 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:50.663338    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:51.299276 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:51.299245    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:45:52.371408 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:45:52.371358    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:45:52.566928 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:52.566885    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:53.395290 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.395256    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"]
Nov 03 21:45:53.400545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.400502    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.413667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.413645    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:53.415485 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.415462    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-27fdf4\"/\"distinct-2-w2-27fdf4-kserve-self-signed-certs\""
Nov 03 21:45:53.415672 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.415543    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-27fdf4\"/\"openshift-service-ca.crt\""
Nov 03 21:45:53.421439 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.421408    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"]
Nov 03 21:45:53.426562 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.426535    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-27fdf4\"/\"kube-root-ca.crt\""
Nov 03 21:45:53.426672 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.426622    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-27fdf4\"/\"default-dockercfg-xk82w\""
Nov 03 21:45:53.431529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.431494    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:53.460404 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.460367    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:53.496582 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.496548    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-home\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.496582 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.496585    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzhxf\" (UniqueName: \"kubernetes.io/projected/7dcb03d9-851b-4209-934c-1d49f68da01b-kube-api-access-hzhxf\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.496806 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.496603    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-dshm\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.496871 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.496849    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-tmp-dir\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.496914 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.496881    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dcb03d9-851b-4209-934c-1d49f68da01b-tls-certs\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.496914 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.496899    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-model-cache\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.624527 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.623625    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-tmp-dir\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.624527 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.623677    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dcb03d9-851b-4209-934c-1d49f68da01b-tls-certs\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.624527 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.623706    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-model-cache\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.624527 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.623821    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-home\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.624527 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.623851    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzhxf\" (UniqueName: \"kubernetes.io/projected/7dcb03d9-851b-4209-934c-1d49f68da01b-kube-api-access-hzhxf\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.638005 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.637968    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-model-cache\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.639203 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.638797    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-home\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.640453 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.640428    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-tmp-dir\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.641050 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.641023    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-dshm\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.661989 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.661911    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzhxf\" (UniqueName: \"kubernetes.io/projected/7dcb03d9-851b-4209-934c-1d49f68da01b-kube-api-access-hzhxf\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.673540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.666055    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-dshm\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.683528 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.678953    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dcb03d9-851b-4209-934c-1d49f68da01b-tls-certs\") pod \"distinct-2-w2-27fdf4-kserve-659645cb79-qnczq\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") " pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:53.716133 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:53.714952    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:54.026484 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:54.026405    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"]
Nov 03 21:45:54.030050 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:45:54.030022    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7dcb03d9_851b_4209_934c_1d49f68da01b.slice/crio-1b5f34a8e3dcb7681c1ad29cd3fb1c61780ea31fbab63c9670cb1b78b37cc122 WatchSource:0}: Error finding container 1b5f34a8e3dcb7681c1ad29cd3fb1c61780ea31fbab63c9670cb1b78b37cc122: Status 404 returned error can't find the container with id 1b5f34a8e3dcb7681c1ad29cd3fb1c61780ea31fbab63c9670cb1b78b37cc122
Nov 03 21:45:54.360544 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:54.360497    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" event={"ID":"7dcb03d9-851b-4209-934c-1d49f68da01b","Type":"ContainerStarted","Data":"0fa6aab340b84bca4bf2ca1354cbf5ecaaac22c67a2083d74d7d774c32ed5968"}
Nov 03 21:45:54.360714 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:54.360554    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" event={"ID":"7dcb03d9-851b-4209-934c-1d49f68da01b","Type":"ContainerStarted","Data":"1b5f34a8e3dcb7681c1ad29cd3fb1c61780ea31fbab63c9670cb1b78b37cc122"}
Nov 03 21:45:54.371443 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:54.371421    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:45:54.396669 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:54.396626    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podStartSLOduration=1.396614354 podStartE2EDuration="1.396614354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:54.396283989 +0000 UTC m=+963.275106273" watchObservedRunningTime="2026-11-03 21:45:54.396614354 +0000 UTC m=+963.275436637"
Nov 03 21:45:54.497780 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:54.497738    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:54.716336 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:54.716258    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:45:55.065816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.065728    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"]
Nov 03 21:45:55.069907 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.069885    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.099640 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.099610    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"]
Nov 03 21:45:55.173035 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.172998    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-data\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.173035 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.173035    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-token\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.173244 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.173152    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-envoy\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.173244 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.173185    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-certs\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.173331 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.173257    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmtht\" (UniqueName: \"kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-kube-api-access-hmtht\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.173331 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.173294    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-credential-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.173331 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.173320    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.173475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.173455    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-podinfo\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.173582 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.173568    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istiod-ca-cert\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.219864 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.219823    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"]
Nov 03 21:45:55.223926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.223902    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.258540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.255577    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"]
Nov 03 21:45:55.275081 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275051    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-envoy\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.275081 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275086    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-certs\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.275301 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275222    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.275301 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275260    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmtht\" (UniqueName: \"kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-kube-api-access-hmtht\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.275420 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275349    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-credential-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.275420 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275389    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-envoy\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.275539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275424    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-data\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.275539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275454    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.275539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275469    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-certs\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.275702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275645    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21f7c684-2bb5-4c67-9009-1d9f854be081-istiod-ca-cert\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.275702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275690    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-token\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.275800 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275720    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-credential-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.275855 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275796    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.275855 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275816    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-credential-socket\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.275855 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.275820    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-podinfo\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.276255 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.276233    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-podinfo\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.276683 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.276657    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istiod-ca-cert\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.277484 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.277225    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istiod-ca-cert\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.277484 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.277346    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-certs\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.277663 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.277486    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-data\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.278006 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.277924    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-token\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.278006 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.277931    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-envoy\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.278006 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.277981    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-data\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.278006 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.277982    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gttzk\" (UniqueName: \"kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-kube-api-access-gttzk\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.279590 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.279571    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-podinfo\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.292791 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.292769    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmtht\" (UniqueName: \"kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-kube-api-access-hmtht\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.295296 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.295276    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-token\") pod \"e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") " pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.378928 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.378884    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-certs\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379110 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.378967    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gttzk\" (UniqueName: \"kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-kube-api-access-gttzk\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379110 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.379035    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379110 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.379073    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-envoy\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379110 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.379098    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-data\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379410 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.379134    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21f7c684-2bb5-4c67-9009-1d9f854be081-istiod-ca-cert\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379410 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.379168    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-token\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379410 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.379195    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-credential-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379410 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.379322    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-certs\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379609 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.379485    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-podinfo\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379609 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.379557    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379714 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.379683    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-data\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379801 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.379782    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-credential-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.379881 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.379864    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21f7c684-2bb5-4c67-9009-1d9f854be081-istiod-ca-cert\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.381575 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.381556    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-envoy\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.381958 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.381931    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-podinfo\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.385719 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.385700    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:55.396494 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.396470    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-token\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.396815 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.396797    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gttzk\" (UniqueName: \"kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-kube-api-access-gttzk\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.537884 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.537854    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:55.696618 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.696539    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"]
Nov 03 21:45:55.700039 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:45:55.700008    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eda1eb4_b61c_44fd_ad6c_d11d122f36ad.slice/crio-be821d1f0f7c4ba66cebbf5dc6fe4d66d5bebbb3f7a2f0b8b2c828ca9d076bae WatchSource:0}: Error finding container be821d1f0f7c4ba66cebbf5dc6fe4d66d5bebbb3f7a2f0b8b2c828ca9d076bae: Status 404 returned error can't find the container with id be821d1f0f7c4ba66cebbf5dc6fe4d66d5bebbb3f7a2f0b8b2c828ca9d076bae
Nov 03 21:45:55.702686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.702653    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:45:55.702758 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.702722    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:45:55.702758 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.702751    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:45:55.838187 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.838153    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"]
Nov 03 21:45:55.841679 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:45:55.841645    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f7c684_2bb5_4c67_9009_1d9f854be081.slice/crio-073d62a54079d2bbe92adb9bffa999bbf4f52b9d408c32c342da4aac5200d26e WatchSource:0}: Error finding container 073d62a54079d2bbe92adb9bffa999bbf4f52b9d408c32c342da4aac5200d26e: Status 404 returned error can't find the container with id 073d62a54079d2bbe92adb9bffa999bbf4f52b9d408c32c342da4aac5200d26e
Nov 03 21:45:55.844393 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.844362    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:45:55.844472 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.844425    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:45:55.844472 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:55.844458    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:45:56.317231 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:56.317192    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:56.397785 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:56.397747    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" event={"ID":"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad","Type":"ContainerStarted","Data":"e2698f86babe330d859ed777b1492d077061f4ef543d0da686d65b17d8950774"}
Nov 03 21:45:56.398242 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:56.397804    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" event={"ID":"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad","Type":"ContainerStarted","Data":"be821d1f0f7c4ba66cebbf5dc6fe4d66d5bebbb3f7a2f0b8b2c828ca9d076bae"}
Nov 03 21:45:56.399372 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:56.399341    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" event={"ID":"21f7c684-2bb5-4c67-9009-1d9f854be081","Type":"ContainerStarted","Data":"2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545"}
Nov 03 21:45:56.399502 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:56.399390    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" event={"ID":"21f7c684-2bb5-4c67-9009-1d9f854be081","Type":"ContainerStarted","Data":"073d62a54079d2bbe92adb9bffa999bbf4f52b9d408c32c342da4aac5200d26e"}
Nov 03 21:45:56.484534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:56.483725    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:56.498086 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:56.498034    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" podStartSLOduration=1.498019438 podStartE2EDuration="1.498019438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:56.459891988 +0000 UTC m=+965.338714273" watchObservedRunningTime="2026-11-03 21:45:56.498019438 +0000 UTC m=+965.376841776"
Nov 03 21:45:56.498391 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:56.498340    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" podStartSLOduration=1.498330226 podStartE2EDuration="1.498330226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:56.497342703 +0000 UTC m=+965.376164987" watchObservedRunningTime="2026-11-03 21:45:56.498330226 +0000 UTC m=+965.377152511"
Nov 03 21:45:56.544963 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:56.539594    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:45:56.545304 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:56.545272    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:45:56.545578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:56.545543    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:45:57.390529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:57.389889    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:45:57.395531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:57.393627    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:45:57.395531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:57.393693    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" podUID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:45:57.539096 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:57.539060    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:45:57.539483 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:57.539121    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:45:58.409538 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:58.408846    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:45:58.409538 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:58.408911    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" podUID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:45:58.538224 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:58.538187    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:45:58.538398 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:58.538238    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:45:58.806589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:58.805885    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:45:59.393924 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:59.393887    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:45:59.394624 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:59.394215    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" podUID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:45:59.541671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:59.539029    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:45:59.541671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:45:59.539127    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:46:00.422534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:00.421350    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:46:00.422534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:00.421412    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" podUID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:46:00.423092 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:00.422916    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:00.538291 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:00.538255    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:46:00.538476 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:00.538316    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:46:01.387532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:01.386889    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:46:01.387532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:01.386954    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" podUID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:46:01.539130 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:01.539088    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:46:01.539531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:01.539160    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:46:01.651673 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:01.651582    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:02.386639 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:02.386595    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:46:02.386827 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:02.386653    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" podUID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:46:02.538724 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:02.538676    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:46:02.538906 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:02.538761    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:46:03.391116 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:03.391086    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:46:03.391618 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:03.391497    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:46:03.392081 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:03.392061    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:46:03.544083 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:03.544043    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:46:03.544489 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:03.544462    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:46:03.545258 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:03.545239    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:46:03.661938 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:03.661763    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:03.715466 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:03.715436    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:46:03.727595 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:03.727573    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:46:03.765078 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:03.765049    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:04.475339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:04.475310    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:46:05.560162 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:05.560122    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:07.462045 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:07.462012    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:07.933659 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:07.933628    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"]
Nov 03 21:46:07.938910 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:07.938887    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:07.956158 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:07.956130    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"]
Nov 03 21:46:07.964010 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:07.963986    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-4eced5\"/\"distinct-2-w1-4eced5-kserve-self-signed-certs\""
Nov 03 21:46:07.985650 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:07.985609    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:08.069339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.069303    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-model-cache\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.069339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.069338    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tls-certs\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.069584 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.069450    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfv8n\" (UniqueName: \"kubernetes.io/projected/063c1a56-0889-40ac-8eca-3f7acb93f7cb-kube-api-access-sfv8n\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.069584 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.069563    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-home\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.069665 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.069603    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-dshm\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.069752 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.069737    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tmp-dir\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.174530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.171455    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tmp-dir\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.174530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.171614    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-model-cache\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.174530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.171641    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tls-certs\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.174530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.171725    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfv8n\" (UniqueName: \"kubernetes.io/projected/063c1a56-0889-40ac-8eca-3f7acb93f7cb-kube-api-access-sfv8n\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.174530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.171818    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-home\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.174530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.171869    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-dshm\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.174530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.173605    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-home\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.174530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.173879    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tmp-dir\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.174530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.174197    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-model-cache\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.184772 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.180380    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tls-certs\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.184772 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.181938    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-dshm\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.202703 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.202662    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfv8n\" (UniqueName: \"kubernetes.io/projected/063c1a56-0889-40ac-8eca-3f7acb93f7cb-kube-api-access-sfv8n\") pod \"distinct-2-w1-4eced5-kserve-dbd886bb7-nq655\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") " pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.258643 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.255610    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:08.559708 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:08.559619    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"]
Nov 03 21:46:08.562810 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:46:08.562780    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod063c1a56_0889_40ac_8eca_3f7acb93f7cb.slice/crio-a8d617baef90a968c09b81b3fb380beae4b5b1349f36f3c35034241146cd384a WatchSource:0}: Error finding container a8d617baef90a968c09b81b3fb380beae4b5b1349f36f3c35034241146cd384a: Status 404 returned error can't find the container with id a8d617baef90a968c09b81b3fb380beae4b5b1349f36f3c35034241146cd384a
Nov 03 21:46:09.264146 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:09.264104    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:09.492579 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:09.492538    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" event={"ID":"063c1a56-0889-40ac-8eca-3f7acb93f7cb","Type":"ContainerStarted","Data":"77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5"}
Nov 03 21:46:09.492769 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:09.492590    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" event={"ID":"063c1a56-0889-40ac-8eca-3f7acb93f7cb","Type":"ContainerStarted","Data":"a8d617baef90a968c09b81b3fb380beae4b5b1349f36f3c35034241146cd384a"}
Nov 03 21:46:09.529208 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:09.529116    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podStartSLOduration=2.52910354 podStartE2EDuration="2.52910354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:46:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:46:09.527853768 +0000 UTC m=+978.406676034" watchObservedRunningTime="2026-11-03 21:46:09.52910354 +0000 UTC m=+978.407925822"
Nov 03 21:46:10.256895 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:10.256862    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:10.989719 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:10.989679    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:12.763833 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:12.763789    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:14.529297 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:14.529252    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:17.122530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:17.121642    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:46:17.122530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:17.121753    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:17.375946 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:17.375836    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:17.498667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:17.498619    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:17.617913 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:17.617851    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:17.744218 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:17.744136    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:17.744640 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:17.744614    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 21:46:18.256138 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:18.256108    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:18.269685 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:18.269648    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:18.614061 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:18.614029    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:20.063557 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:20.063496    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:21.961569 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:21.961525    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:22.372999 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:46:22.372965    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:46:23.653863 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:23.653807    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:25.444635 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:25.444600    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:27.160426 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:27.160388    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:28.943366 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:28.943326    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:30.755115 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:30.755072    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:32.603759 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:32.603729    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:34.269469 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:34.269435    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:34.987490 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:34.987459    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"]
Nov 03 21:46:34.992912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:34.992893    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.003950 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.002163    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"]
Nov 03 21:46:35.006199 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.006178    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.024160 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.024129    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"]
Nov 03 21:46:35.033244 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.033220    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"]
Nov 03 21:46:35.114538 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.114482    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-podinfo\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.114538 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.114544    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-certs\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.114814 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.114716    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-data\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.114814 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.114757    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-data\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.114888 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.114850    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-certs\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.114888 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.114883    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-envoy\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.114988 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.114900    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-podinfo\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.114988 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.114949    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.115053 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.115014    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-token\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.115053 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.115049    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-credential-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.115154 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.115081    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-credential-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.115154 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.115130    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istiod-ca-cert\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.115251 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.115198    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-token\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.115251 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.115239    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5nqr\" (UniqueName: \"kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-kube-api-access-l5nqr\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.115334 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.115261    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz5nv\" (UniqueName: \"kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-kube-api-access-fz5nv\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.115334 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.115279    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a09566e5-58cc-4bb7-bf49-9912a8555677-istiod-ca-cert\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.115334 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.115315    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-envoy\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.115453 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.115361    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.217061 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217031    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-credential-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.217247 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217078    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-credential-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.217247 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217096    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istiod-ca-cert\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.217247 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217133    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-token\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.217247 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217151    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5nqr\" (UniqueName: \"kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-kube-api-access-l5nqr\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.217247 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217171    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz5nv\" (UniqueName: \"kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-kube-api-access-fz5nv\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.217247 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217191    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a09566e5-58cc-4bb7-bf49-9912a8555677-istiod-ca-cert\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.217247 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217211    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-envoy\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.217761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217284    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.217761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217347    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-podinfo\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.217761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217377    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-certs\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.217761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217458    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-credential-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.217761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217484    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-data\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.217761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217533    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-data\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.217761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217556    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-credential-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.217761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217594    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-certs\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.217761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217632    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-envoy\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.217761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217654    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-podinfo\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.217761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217682    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.217761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217728    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-token\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.219190 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217838    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-data\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.219190 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217941    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-certs\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.219190 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.217962    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istiod-ca-cert\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.219190 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.218179    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a09566e5-58cc-4bb7-bf49-9912a8555677-istiod-ca-cert\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.219190 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.218244    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-data\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.219190 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.218245    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.219190 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.218423    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-socket\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.219190 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.218448    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-certs\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.220071 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.220049    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-envoy\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.220153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.220085    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-envoy\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.220189 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.220158    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-podinfo\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.220417 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.220400    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-podinfo\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.235193 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.235168    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz5nv\" (UniqueName: \"kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-kube-api-access-fz5nv\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.235288 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.235232    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-token\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.235595 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.235574    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5nqr\" (UniqueName: \"kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-kube-api-access-l5nqr\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.245918 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.245870    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-token\") pod \"e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") " pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.309107 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.309081    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:35.322070 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.322043    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:35.635535 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.635489    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"]
Nov 03 21:46:35.638726 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.638699    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"]
Nov 03 21:46:35.639128 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:46:35.639106    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23ca699_38c0_44f3_bd17_5bc17dcf1963.slice/crio-60c327f787f67c6f573539476d20352f99f53fb02d0e7096d729bfe537fb9907 WatchSource:0}: Error finding container 60c327f787f67c6f573539476d20352f99f53fb02d0e7096d729bfe537fb9907: Status 404 returned error can't find the container with id 60c327f787f67c6f573539476d20352f99f53fb02d0e7096d729bfe537fb9907
Nov 03 21:46:35.641877 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:46:35.641855    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda09566e5_58cc_4bb7_bf49_9912a8555677.slice/crio-9cf7d73a8885d59ae7d568042e5412fbf6a2eeaf95ae0cebe24561648f59132b WatchSource:0}: Error finding container 9cf7d73a8885d59ae7d568042e5412fbf6a2eeaf95ae0cebe24561648f59132b: Status 404 returned error can't find the container with id 9cf7d73a8885d59ae7d568042e5412fbf6a2eeaf95ae0cebe24561648f59132b
Nov 03 21:46:35.642091 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.642060    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:35.642147 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.642132    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:35.642193 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.642179    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:35.644381 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.644334    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:35.644433 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.644384    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:35.644433 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.644417    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:35.707813 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.707779    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" event={"ID":"c23ca699-38c0-44f3-bd17-5bc17dcf1963","Type":"ContainerStarted","Data":"60c327f787f67c6f573539476d20352f99f53fb02d0e7096d729bfe537fb9907"}
Nov 03 21:46:35.709477 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.709450    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" event={"ID":"a09566e5-58cc-4bb7-bf49-9912a8555677","Type":"ContainerStarted","Data":"9cf7d73a8885d59ae7d568042e5412fbf6a2eeaf95ae0cebe24561648f59132b"}
Nov 03 21:46:35.941235 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:35.941144    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:36.042533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:36.042280    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:36.537038 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:36.536999    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:36.719800 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:36.719767    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" event={"ID":"c23ca699-38c0-44f3-bd17-5bc17dcf1963","Type":"ContainerStarted","Data":"c4f29fa756b548bbb7340e39d7f3e2b539d1a88d86a30dc5b05c4c9a390ca992"}
Nov 03 21:46:36.721176 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:36.721154    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" event={"ID":"a09566e5-58cc-4bb7-bf49-9912a8555677","Type":"ContainerStarted","Data":"6150896afac42fe08e1f666392c6fa92fa243494143208a32f04855fe326acf2"}
Nov 03 21:46:36.834996 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:36.834894    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" podStartSLOduration=2.834878549 podStartE2EDuration="2.834878549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:46:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:46:36.790891228 +0000 UTC m=+1005.669713534" watchObservedRunningTime="2026-11-03 21:46:36.834878549 +0000 UTC m=+1005.713700832"
Nov 03 21:46:36.837149 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:36.837101    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" podStartSLOduration=2.837091448 podStartE2EDuration="2.837091448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:46:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:46:36.833865156 +0000 UTC m=+1005.712687466" watchObservedRunningTime="2026-11-03 21:46:36.837091448 +0000 UTC m=+1005.715913728"
Nov 03 21:46:37.309668 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:37.309631    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:37.311467 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:37.311437    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 21:46:37.311690 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:37.311657    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 21:46:37.323085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:37.323064    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:37.324305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:37.324272    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:46:37.324409 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:37.324324    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:46:37.893702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:37.893666    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:38.310589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:38.309872    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 21:46:38.310589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:38.309936    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 21:46:38.323222 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:38.323194    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:46:38.323357 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:38.323237    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:46:39.310566 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:39.310394    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 21:46:39.310566 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:39.310455    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 21:46:39.323085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:39.323033    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:46:39.323242 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:39.323115    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:46:39.824532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:39.824487    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:40.039364 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.039332    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"]
Nov 03 21:46:40.044961 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.044934    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.056007 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.055969    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-podinfo\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.056160 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.056036    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-envoy\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.056160 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.056072    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istiod-ca-cert\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.056160 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.056141    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-data\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.056327 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.056193    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-token\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.056327 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.056254    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-certs\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.056327 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.056300    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.056465 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.056368    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-credential-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.056465 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.056390    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lbqd\" (UniqueName: \"kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-kube-api-access-9lbqd\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.061464 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.060959    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"]
Nov 03 21:46:40.069876 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.069849    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.135678 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.135646    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"]
Nov 03 21:46:40.174532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.160382    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"]
Nov 03 21:46:40.226542 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.226492    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-podinfo\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.226834 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.226808    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-envoy\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.227141 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.226973    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnd5r\" (UniqueName: \"kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-kube-api-access-vnd5r\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.227293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.227278    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istiod-ca-cert\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.227605 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.227421    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.227747 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.227733    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-credential-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.227991 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.227842    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-envoy\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.228147 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.228128    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-data\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.228302 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.228286    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-data\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.228668 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.228638    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-token\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.228825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.228808    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-certs\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.228992 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.228975    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-certs\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.229147 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.229128    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8126051-d2b3-488f-86d0-a2919329a4db-istio-podinfo\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.229274 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.229259    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.229408 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.229394    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-istio-token\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.229951 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.229691    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-credential-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.230085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.230070    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lbqd\" (UniqueName: \"kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-kube-api-access-9lbqd\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.230213 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.230187    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8126051-d2b3-488f-86d0-a2919329a4db-istiod-ca-cert\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.231559 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.231434    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-data\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.231687 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.231457    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-certs\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.232001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.231943    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.232085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.229310    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istiod-ca-cert\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.232310 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.232266    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-credential-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.236556 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.235759    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-podinfo\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.240904 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.240882    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-envoy\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.253668 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.253639    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-token\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.253972 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.253953    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lbqd\" (UniqueName: \"kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-kube-api-access-9lbqd\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.309899 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.309864    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 21:46:40.310155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.310129    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 21:46:40.322376 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.322340    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:46:40.322750 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.322390    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:46:40.333858 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.332785    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8126051-d2b3-488f-86d0-a2919329a4db-istio-podinfo\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.334146 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.334124    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-istio-token\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.335529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.335496    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8126051-d2b3-488f-86d0-a2919329a4db-istio-podinfo\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.335843 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.335820    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8126051-d2b3-488f-86d0-a2919329a4db-istiod-ca-cert\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.335954 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.335829    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8126051-d2b3-488f-86d0-a2919329a4db-istiod-ca-cert\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.336660 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.335976    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnd5r\" (UniqueName: \"kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-kube-api-access-vnd5r\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.338188 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.337758    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.338188 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.337789    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-credential-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.338188 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.338117    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-credential-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.338188 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.338098    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-socket\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.338188 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.338140    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-envoy\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.338188 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.338180    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-data\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.339180 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.338312    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-certs\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.339180 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.338665    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-certs\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.339180 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.338940    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-data\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.340395 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.339622    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:46:40.340546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.340493    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-envoy\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.344854 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.344831    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"]
Nov 03 21:46:40.350899 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.350844    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.439042 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.439008    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-istio-token\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.440068 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.439716    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istiod-ca-cert\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.440370 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.440348    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-data\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.440696 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.440675    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-envoy\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.440855 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.440836    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.440964 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.440949    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-token\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.442070 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.441292    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbxf6\" (UniqueName: \"kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-kube-api-access-pbxf6\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.442070 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.441459    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-certs\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.442070 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.441489    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-podinfo\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.442070 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.441641    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-credential-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.444172 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.444146    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"]
Nov 03 21:46:40.444867 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.444839    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnd5r\" (UniqueName: \"kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-kube-api-access-vnd5r\") pod \"e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") " pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.522690 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.522652    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:40.542716 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.542684    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-certs\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.542825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.542721    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-podinfo\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.542825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.542777    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-credential-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.542825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.542820    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istiod-ca-cert\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.542995 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.542859    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-data\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.542995 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.542934    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-envoy\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.542995 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.542957    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.542995 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.542974    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-token\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.542995 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.542994    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbxf6\" (UniqueName: \"kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-kube-api-access-pbxf6\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.543250 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.543029    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-certs\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.543372 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.543349    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-credential-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.543683 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.543657    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-data\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.543803 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.543782    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-socket\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.543898 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.543684    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istiod-ca-cert\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.544678 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.544658    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:40.546817 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.546790    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-podinfo\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.547124 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.547101    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-envoy\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.577586 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.577193    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-token\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.593206 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.593173    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbxf6\" (UniqueName: \"kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-kube-api-access-pbxf6\") pod \"e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") " pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.734318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.734049    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:40.936556 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:46:40.933956    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32c16e9_42d8_4d67_85d2_42826a1ff1b1.slice/crio-9c6d95c115389619fcb4cc5276ae43a80d979d068e86dc09ba4a7c527a9e8fd2 WatchSource:0}: Error finding container 9c6d95c115389619fcb4cc5276ae43a80d979d068e86dc09ba4a7c527a9e8fd2: Status 404 returned error can't find the container with id 9c6d95c115389619fcb4cc5276ae43a80d979d068e86dc09ba4a7c527a9e8fd2
Nov 03 21:46:40.938846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.938234    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:40.938846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.938323    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:40.938846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.938374    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:40.941168 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:40.941139    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"]
Nov 03 21:46:41.032305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:41.032269    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"]
Nov 03 21:46:41.040032 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:41.039995    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:41.040154 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:41.040069    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:41.040154 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:41.040100    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:41.240351 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:46:41.239716    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad15d2cf_954f_4711_a95f_5ce2a24fd12f.slice/crio-d9a16cf8533d06298f315ff9a2ca2ecb12d3ed29a43a018f8249356e8d0b715e WatchSource:0}: Error finding container d9a16cf8533d06298f315ff9a2ca2ecb12d3ed29a43a018f8249356e8d0b715e: Status 404 returned error can't find the container with id d9a16cf8533d06298f315ff9a2ca2ecb12d3ed29a43a018f8249356e8d0b715e
Nov 03 21:46:41.260097 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:41.260075    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"]
Nov 03 21:46:41.260607 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:41.260497    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:41.260801 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:41.260780    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:41.260962 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:41.260941    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:41.329534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:41.312905    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 21:46:41.329534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:41.312966    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 21:46:41.330355 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:41.330149    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:46:41.330355 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:41.330205    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:46:42.010885 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.010831    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" event={"ID":"e8126051-d2b3-488f-86d0-a2919329a4db","Type":"ContainerStarted","Data":"d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2"}
Nov 03 21:46:42.011167 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.011140    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" event={"ID":"e8126051-d2b3-488f-86d0-a2919329a4db","Type":"ContainerStarted","Data":"d41bcdc16a31180f8a583476e0088eb6193600a981623e8174f59d79c1d7bd7d"}
Nov 03 21:46:42.021307 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.021230    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" event={"ID":"ad15d2cf-954f-4711-a95f-5ce2a24fd12f","Type":"ContainerStarted","Data":"d9a16cf8533d06298f315ff9a2ca2ecb12d3ed29a43a018f8249356e8d0b715e"}
Nov 03 21:46:42.043343 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.043302    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" event={"ID":"b32c16e9-42d8-4d67-85d2-42826a1ff1b1","Type":"ContainerStarted","Data":"8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e"}
Nov 03 21:46:42.043601 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.043579    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" event={"ID":"b32c16e9-42d8-4d67-85d2-42826a1ff1b1","Type":"ContainerStarted","Data":"9c6d95c115389619fcb4cc5276ae43a80d979d068e86dc09ba4a7c527a9e8fd2"}
Nov 03 21:46:42.077531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.073809    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" podStartSLOduration=2.073788157 podStartE2EDuration="2.073788157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:46:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:46:42.072972114 +0000 UTC m=+1010.951794424" watchObservedRunningTime="2026-11-03 21:46:42.073788157 +0000 UTC m=+1010.952610441"
Nov 03 21:46:42.175592 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.153974    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" podStartSLOduration=2.153954866 podStartE2EDuration="2.153954866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:46:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:46:42.149331435 +0000 UTC m=+1011.028153751" watchObservedRunningTime="2026-11-03 21:46:42.153954866 +0000 UTC m=+1011.032777132"
Nov 03 21:46:42.322532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.321007    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 21:46:42.322532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.321094    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 21:46:42.326533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.325666    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:46:42.326533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.325717    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:46:42.361480 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.361433    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:42.381025 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.380981    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:42.525016 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.524932    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:42.528676 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.528637    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 21:46:42.528931 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.528886    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 21:46:42.550919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.550886    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:42.551753 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.551720    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:46:42.552091 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.552065    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:46:42.606032 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.605974    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 03 21:46:42.619555 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.619400    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:42.762674 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.762618    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:42.951675 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.951621    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:42.994740 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:42.994678    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:43.073794 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.073758    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" event={"ID":"ad15d2cf-954f-4711-a95f-5ce2a24fd12f","Type":"ContainerStarted","Data":"ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d"}
Nov 03 21:46:43.128534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.128272    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" podStartSLOduration=3.128250363 podStartE2EDuration="3.128250363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:46:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:46:43.124923393 +0000 UTC m=+1012.003745678" watchObservedRunningTime="2026-11-03 21:46:43.128250363 +0000 UTC m=+1012.007072649"
Nov 03 21:46:43.333981 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.333882    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 21:46:43.334242 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.334207    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 21:46:43.334941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.334920    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:46:43.335102 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.335082    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:46:43.382629 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.382582    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:43.383491 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.383467    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:46:43.679543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.628599    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:43.679543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.654993    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:46:43.790533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.786152    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:46:43.790533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.786215    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:46:43.886529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.843608    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:43.886529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.872805    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 21:46:43.886529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.872863    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 21:46:43.886529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.881851    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:43.894533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.889299    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:46:43.917685 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.902633    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:43.946631 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.930149    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:46:43.946631 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.930216    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:46:43.971542 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.952630    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Nov 03 21:46:43.971542 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.963723    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:46:44.000604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.995564    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:46:44.000604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:43.997549    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:44.014539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.010595    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:44.014539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.011753    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:46:44.483549 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.455004    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 21:46:44.483549 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.455092    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 21:46:44.483549 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.456637    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:44.483549 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.456939    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:46:44.483549 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.456975    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:46:44.521592 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.512134    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Nov 03 21:46:44.521592 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.517260    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": context deadline exceeded"
Nov 03 21:46:44.564085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.563949    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 21:46:44.564085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.564008    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 21:46:44.617189 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.587168    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:46:44.617189 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.587223    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:46:44.673560 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.646624    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": context deadline exceeded"
Nov 03 21:46:44.703249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.703194    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:44.737537 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.737415    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:46:44.747431 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.746647    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:46:44.845613 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.838140    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:44.879557 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.864040    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:44.908534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.883623    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:44.909148 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.898622    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 21:46:44.917530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.909614    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:46:44.957573 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:44.955774    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:45.008605 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.008442    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:45.009256 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.009235    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:46:45.011994 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.011951    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:45.032656 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.021205    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:45.032656 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.021934    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:46:45.596634 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.596586    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:45.607902 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.607859    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/health\": context deadline exceeded"
Nov 03 21:46:45.877822 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.610357    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 21:46:46.172940 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.172827    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 21:46:46.173238 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.763640    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:46.215534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.190177    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": context deadline exceeded"
Nov 03 21:46:46.250540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.918377    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": context deadline exceeded"
Nov 03 21:46:46.250540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.052558    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Nov 03 21:46:46.250540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.052633    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 03 21:46:46.250540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.740832    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:46.250540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.690382    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:46:46.250540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.235822    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:46:46.250540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.193926    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:46.250540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.916609    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:46.250540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:45.916728    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:46:46.250540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.236339    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:46:46.360533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.287105    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Nov 03 21:46:46.502593 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.482667    4454 patch_prober.go:29] interesting pod/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:46:46.502593 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.482758    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:46.679943 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.679897    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:46.864627 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.847044    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:46:46.864627 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.847101    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:46:46.864627 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:46:46.861099    4454 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.289s"
Nov 03 21:46:46.864627 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.861189    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:46:46.864627 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.861247    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:46:46.864627 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.861283    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:46.878528 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.869605    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": context deadline exceeded"
Nov 03 21:46:46.922629 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.911736    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 21:46:46.922629 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.911850    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 21:46:46.930657 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:46.930615    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:47.108120 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.108081    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:47.158586 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.140882    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:47.159249 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.159212    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:46:47.193542 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.177639    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:47.256915 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.190906    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:46:47.257170 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.257136    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:46:47.315536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.282183    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:47.315536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.283413    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:47.448589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.364551    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:46:47.539817 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.539779    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 21:46:47.540080 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.540053    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 21:46:47.540194 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.410699    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:47.540752 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.410682    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:47.541097 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.410730    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:47.541419 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.410749    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:47.541873 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.410767    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:47.542232 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.410799    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:47.542583 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.456323    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:47.548320 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.410645    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:47.653535 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.558935    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:46:47.653535 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.620968    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:46:47.671539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.666096    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:46:47.725696 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.725590    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Nov 03 21:46:47.749296 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.749258    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:46:47.749611 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.749553    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:46:47.934203 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.933140    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:47.934790 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:47.934501    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:46:48.057884 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.054430    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:46:48.344376 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.343646    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": context deadline exceeded"
Nov 03 21:46:48.563620 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.546847    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Nov 03 21:46:48.563620 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.547091    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 03 21:46:48.563620 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.547277    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Nov 03 21:46:48.563620 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.547441    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 21:46:48.566233 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.566212    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:46:48.620987 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.580994    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 21:46:48.620987 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.581061    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 21:46:48.620987 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.581213    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Nov 03 21:46:48.620987 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.614442    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:46:48.620987 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.614494    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:46:48.705315 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.705287    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:46:48.829977 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.823452    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:46:48.830633 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.830602    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:46:48.977545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:48.977430    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:49.056640 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.056591    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:49.180544 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.151449    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:46:49.359762 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.359604    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": context deadline exceeded"
Nov 03 21:46:49.456376 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.456241    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:49.561537 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.545202    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 21:46:49.587531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.564586    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 21:46:49.765580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.673715    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:49.765580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.676308    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:49.765580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.676842    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:46:49.765580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.676921    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:46:49.765580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.677026    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 03 21:46:49.765580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.754255    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:46:49.765580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.754353    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:46:49.853537 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.838823    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:46:49.952561 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.952533    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:46:49.952896 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.952877    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:46:49.953098 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:49.953085    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:46:50.164589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:50.083551    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:50.164589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:50.108328    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:46:50.531583 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:50.499611    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:50.567212 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:50.567176    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 21:46:50.567478 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:50.567448    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 21:46:50.580714 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:50.580680    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:46:50.580965 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:50.580937    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:46:50.581932 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:50.581912    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:46:50.709528 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:50.705605    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:46:50.755449 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:50.755405    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:46:50.755653 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:50.755471    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:46:50.846976 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:50.846949    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:46:51.523916 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:51.523877    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 21:46:51.524313 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:51.523947    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 21:46:51.545012 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:51.544975    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:46:51.545174 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:51.545040    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:46:51.735149 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:51.735113    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:46:51.735338 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:51.735188    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:46:52.393674 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:46:52.393637    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:46:52.528800 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:52.528766    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:52.529227 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:52.529203    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:52.530315 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:52.530290    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:46:52.560914 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:52.559996    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:52.560914 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:52.560722    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:52.561702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:52.561676    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:46:52.740672 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:52.740587    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:52.741049 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:52.740999    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:52.741768 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:52.741751    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:46:53.252313 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:53.252279    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:55.647387 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.647341    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"]
Nov 03 21:46:55.653541 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.653476    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.680704 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.680675    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"]
Nov 03 21:46:55.753342 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.753313    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-envoy\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.753499 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.753361    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-data\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.753499 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.753468    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-socket\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.753628 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.753607    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-token\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.753660 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.753652    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-podinfo\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.753696 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.753670    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75796449-4fc2-4abb-aa29-0435f1464bd6-istiod-ca-cert\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.753731 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.753696    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6lkj\" (UniqueName: \"kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-kube-api-access-h6lkj\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.753826 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.753809    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-certs\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.753890 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.753878    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-credential-socket\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.855798 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.855764    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-credential-socket\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.855975 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.855826    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-envoy\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.855975 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.855871    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-data\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.855975 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.855927    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-socket\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.856120 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.856023    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-token\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.856120 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.856073    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-podinfo\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.856120 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.856099    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75796449-4fc2-4abb-aa29-0435f1464bd6-istiod-ca-cert\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.856271 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.856138    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6lkj\" (UniqueName: \"kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-kube-api-access-h6lkj\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.856271 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.856225    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-certs\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.856271 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.856235    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-credential-socket\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.856478 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.856454    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-socket\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.856628 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.856610    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-data\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.856715 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.856612    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-certs\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.857083 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.857059    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75796449-4fc2-4abb-aa29-0435f1464bd6-istiod-ca-cert\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.858325 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.858302    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-envoy\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.858760 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.858739    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-podinfo\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.873850 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.873827    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-token\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.874085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.874064    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6lkj\" (UniqueName: \"kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-kube-api-access-h6lkj\") pod \"e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") " pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:55.967925 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:55.967834    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:56.250605 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:56.235043    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:56.279884 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:56.279860    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"]
Nov 03 21:46:56.282349 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:46:56.282320    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75796449_4fc2_4abb_aa29_0435f1464bd6.slice/crio-c898ae86afb2f806c04abe253c8b3ced3e437fc8be3c718c3ed92ba72f1794c7 WatchSource:0}: Error finding container c898ae86afb2f806c04abe253c8b3ced3e437fc8be3c718c3ed92ba72f1794c7: Status 404 returned error can't find the container with id c898ae86afb2f806c04abe253c8b3ced3e437fc8be3c718c3ed92ba72f1794c7
Nov 03 21:46:56.284751 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:56.284718    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:56.284839 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:56.284778    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:56.284839 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:56.284809    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:46:57.083411 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:57.083375    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" event={"ID":"75796449-4fc2-4abb-aa29-0435f1464bd6","Type":"ContainerStarted","Data":"f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3"}
Nov 03 21:46:57.083884 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:57.083431    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" event={"ID":"75796449-4fc2-4abb-aa29-0435f1464bd6","Type":"ContainerStarted","Data":"c898ae86afb2f806c04abe253c8b3ced3e437fc8be3c718c3ed92ba72f1794c7"}
Nov 03 21:46:57.137284 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:57.137218    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" podStartSLOduration=2.137198633 podStartE2EDuration="2.137198633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:46:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:46:57.13384789 +0000 UTC m=+1026.012670194" watchObservedRunningTime="2026-11-03 21:46:57.137198633 +0000 UTC m=+1026.016020917"
Nov 03 21:46:57.982529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:57.970852    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:46:57.982529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:57.974617    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:46:57.982529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:57.974686    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:46:58.647579 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:58.647543    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:46:58.977589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:58.968865    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:46:58.977589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:58.968925    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:46:59.981530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:59.970875    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:46:59.981530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:46:59.970951    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:47:00.389070 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:00.389032    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:00.998809 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:00.998769    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:47:00.999299 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:00.998846    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:47:01.976531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:01.975534    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:47:01.976531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:01.975608    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:47:02.970809 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:02.970769    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:47:02.971435 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:02.971393    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:47:03.245992 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:03.244791    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:03.257647 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:03.257613    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:03.628128 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:03.628071    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:03.653630 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:03.653589    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:03.871578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:03.870567    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:03.871578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:03.870867    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 03 21:47:03.872526 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:03.872141    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:04.032139 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:04.032048    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:47:04.032742 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:04.032708    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:47:04.265686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:04.265627    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 21:47:04.269530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:04.266371    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:47:04.344028 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:04.343935    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:04.426403 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:04.426361    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:04.459034 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:04.458990    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:04.543297 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:04.543267    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:47:05.239810 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:05.239780    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:47:05.240303 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:05.240163    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:47:05.240906 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:05.240883    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:47:07.287699 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:07.287669    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:09.281671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:09.281632    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:10.883121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.883090    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"]
Nov 03 21:47:10.887433 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.887413    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:10.917092 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.917064    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"]
Nov 03 21:47:10.957717 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.957688    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-credential-socket\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:10.957885 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.957753    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-certs\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:10.957885 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.957834    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2cq8\" (UniqueName: \"kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-kube-api-access-v2cq8\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:10.958015 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.957893    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-podinfo\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:10.958079 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.958055    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-socket\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:10.958225 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.958203    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-envoy\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:10.958277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.958241    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-data\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:10.958277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.958270    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-token\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:10.958444 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.958384    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istiod-ca-cert\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:10.969060 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:10.969011    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:11.067558 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.063235    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-socket\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.075407 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.075144    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-envoy\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.075407 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.075198    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-data\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.075407 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.075237    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-token\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.075407 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.075286    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istiod-ca-cert\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.075407 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.075353    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-credential-socket\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.075832 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.075443    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-certs\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.075832 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.075497    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2cq8\" (UniqueName: \"kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-kube-api-access-v2cq8\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.075832 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.075682    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-podinfo\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.088475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.076632    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-socket\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.088475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.078542    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-certs\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.088475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.079463    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-credential-socket\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.088475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.079566    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-data\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.088475 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.080174    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istiod-ca-cert\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.157754 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.157559    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2cq8\" (UniqueName: \"kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-kube-api-access-v2cq8\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.164176 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.164139    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-envoy\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.164602 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.164574    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-token\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.165640 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.165614    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-podinfo\") pod \"e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") " pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.202924 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.202892    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:11.520363 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.520272    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"]
Nov 03 21:47:11.524200 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:47:11.524172    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b54323_d6b0_40b7_ba3f_97ea85d99d18.slice/crio-8439e4b8388964794519899b7cee743e9c77cd31df1115da52e0bc30e4185797 WatchSource:0}: Error finding container 8439e4b8388964794519899b7cee743e9c77cd31df1115da52e0bc30e4185797: Status 404 returned error can't find the container with id 8439e4b8388964794519899b7cee743e9c77cd31df1115da52e0bc30e4185797
Nov 03 21:47:11.526874 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.526840    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:47:11.526950 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.526904    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:47:11.526950 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:11.526933    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:47:12.354901 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:12.354855    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx" event={"ID":"28b54323-d6b0-40b7-ba3f-97ea85d99d18","Type":"ContainerStarted","Data":"9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b"}
Nov 03 21:47:12.355346 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:12.354921    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx" event={"ID":"28b54323-d6b0-40b7-ba3f-97ea85d99d18","Type":"ContainerStarted","Data":"8439e4b8388964794519899b7cee743e9c77cd31df1115da52e0bc30e4185797"}
Nov 03 21:47:12.424389 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:12.424337    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx" podStartSLOduration=2.424321866 podStartE2EDuration="2.424321866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:47:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:47:12.423270317 +0000 UTC m=+1041.302092621" watchObservedRunningTime="2026-11-03 21:47:12.424321866 +0000 UTC m=+1041.303144149"
Nov 03 21:47:12.752943 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:12.752862    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:13.204059 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:13.204022    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:13.205502 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:13.205477    4454 patch_prober.go:29] interesting pod/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 21:47:13.205650 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:13.205540    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx" podUID="28b54323-d6b0-40b7-ba3f-97ea85d99d18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 21:47:14.204062 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:14.204025    4454 patch_prober.go:29] interesting pod/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 21:47:14.204470 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:14.204092    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx" podUID="28b54323-d6b0-40b7-ba3f-97ea85d99d18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 21:47:14.589462 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:14.586485    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:15.203171 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:15.203135    4454 patch_prober.go:29] interesting pod/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 21:47:15.203359 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:15.203194    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx" podUID="28b54323-d6b0-40b7-ba3f-97ea85d99d18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 21:47:16.203706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:16.203670    4454 patch_prober.go:29] interesting pod/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 21:47:16.204133 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:16.203730    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx" podUID="28b54323-d6b0-40b7-ba3f-97ea85d99d18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 21:47:16.321206 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:16.321162    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:17.203615 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:17.203580    4454 patch_prober.go:29] interesting pod/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 21:47:17.203788 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:17.203632    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx" podUID="28b54323-d6b0-40b7-ba3f-97ea85d99d18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 21:47:17.276092 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:17.270880    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:18.187074 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:18.187031    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:18.694696 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:18.694659    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:18.695907 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:18.695879    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:18.697451 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:18.697429    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:47:20.066115 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:20.066083    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:20.729420 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:20.729379    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:47:22.314108 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:22.314065    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:22.421154 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:22.421108    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:22.696528 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:47:22.694198    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:47:22.707671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:22.707630    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:22.963722 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:22.963604    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:23.320791 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:23.319599    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:23.478533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:23.470597    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:23.574530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:23.574413    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 21:47:23.575407 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:23.575377    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:23.582604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:23.575583    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:23.680530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:23.679605    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:23.726532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:23.725955    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": context deadline exceeded"
Nov 03 21:47:23.726738 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:23.726648    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:47:23.751532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:23.728226    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:23.852647 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:23.852597    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": context deadline exceeded"
Nov 03 21:47:23.972897 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:23.972722    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Nov 03 21:47:23.974682 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:23.973419    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:47:24.321589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.320595    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:24.322072 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.321831    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:47:24.595846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.595805    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:24.736532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.711137    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:47:24.736532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.599070    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Nov 03 21:47:24.736532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.615813    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": context deadline exceeded"
Nov 03 21:47:24.736532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.621327    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:24.736532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.712689    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:47:24.736532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.712744    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:47:24.736532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.640586    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:24.736532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.688617    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:24.736532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.714497    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:47:24.737231 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.736634    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:24.737231 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.736924    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:24.737231 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.736988    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": context deadline exceeded"
Nov 03 21:47:24.738284 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.737433    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:47:24.739345 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.615780    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:24.739756 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.739735    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:47:24.860845 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.860748    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:24.869307 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.869277    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:47:24.981466 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:24.981413    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:25.304613 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:25.248705    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:25.391643 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:25.320784    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:25.632044 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:25.631999    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:25.712022 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:25.711969    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:25.712968 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:25.712936    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:25.738309 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:25.713151    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:25.739029 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:25.736595    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:25.739424 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:25.737927    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:25.748168 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:25.737949    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:47:25.878532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:25.867872    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:26.119130 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:26.119070    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:26.119424 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:26.119390    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:26.466593 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:26.441875    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:26.469422 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:26.469379    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:26.799937 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:26.799827    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 21:47:26.820718 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:26.820663    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Nov 03 21:47:26.832591 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:26.825905    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 03 21:47:26.838062 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:26.838002    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": context deadline exceeded"
Nov 03 21:47:26.897184 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:26.859160    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:47:26.898630 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:26.886995    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:26.900466 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:26.900427    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:27.102902 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.102869    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"]
Nov 03 21:47:27.126362 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.126316    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:27.166574 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.163437    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:27.339541 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.338736    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.495588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.491610    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45nrw\" (UniqueName: \"kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-kube-api-access-45nrw\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.495588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.491783    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-certs\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.495588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.491826    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-envoy\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.495588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.491851    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-data\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.495588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.491891    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istiod-ca-cert\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.495588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.491954    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-token\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.495588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.492070    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-credential-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.495588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.492135    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.495588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.492222    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-podinfo\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.614562 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.606796    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:27.706792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.659559    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-credential-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.706792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.659642    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.706792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.659728    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-podinfo\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.706792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.659769    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45nrw\" (UniqueName: \"kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-kube-api-access-45nrw\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.706792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.659916    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-certs\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.706792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.659951    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-envoy\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.706792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.659975    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-data\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.706792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.660013    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istiod-ca-cert\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.706792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.660076    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-token\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.706792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.660997    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-credential-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.706792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.661270    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.794502 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.794467    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-token\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.928539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.893815    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-data\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.948849 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.917103    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istiod-ca-cert\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.980889 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.917404    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-certs\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:27.981141 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.917458    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:27.981369 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.929897    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:27.981839 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.980793    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-envoy\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:28.122529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.120242    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"]
Nov 03 21:47:28.122529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.120341    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:47:28.122529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.120379    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:47:28.122529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.120409    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:47:28.122529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.120441    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:47:28.122529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.120474    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:47:28.127759 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.123844    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:47:28.129904 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.129845    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:28.136981 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:27.917499    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:47:28.152217 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.152193    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-podinfo\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:28.153162 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.153138    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:47:28.153916 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.153868    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45nrw\" (UniqueName: \"kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-kube-api-access-45nrw\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:28.170327 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.167100    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:47:28.170327 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.169011    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:47:28.181788 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.181770    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:28.187809 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.187789    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:47:28.524285 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.524200    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"]
Nov 03 21:47:28.536604 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:47:28.535781    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c5a0bd_9878_4b3b_9f7d_e047754c5c36.slice/crio-28259d7b640d193f3351251d688b652f9846a6c7c11e4e593886ea517236d2cb WatchSource:0}: Error finding container 28259d7b640d193f3351251d688b652f9846a6c7c11e4e593886ea517236d2cb: Status 404 returned error can't find the container with id 28259d7b640d193f3351251d688b652f9846a6c7c11e4e593886ea517236d2cb
Nov 03 21:47:28.539347 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.539309    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:47:28.539876 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.539852    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:47:28.540039 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:28.540014    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:47:29.078588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:29.078544    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" event={"ID":"93c5a0bd-9878-4b3b-9f7d-e047754c5c36","Type":"ContainerStarted","Data":"bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b"}
Nov 03 21:47:29.078812 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:29.078647    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" event={"ID":"93c5a0bd-9878-4b3b-9f7d-e047754c5c36","Type":"ContainerStarted","Data":"28259d7b640d193f3351251d688b652f9846a6c7c11e4e593886ea517236d2cb"}
Nov 03 21:47:29.127803 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:29.127756    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" podStartSLOduration=4.127743764 podStartE2EDuration="4.127743764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:47:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:47:29.127206362 +0000 UTC m=+1058.006028648" watchObservedRunningTime="2026-11-03 21:47:29.127743764 +0000 UTC m=+1058.006566047"
Nov 03 21:47:29.185731 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:29.185697    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:29.187261 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:29.187235    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 21:47:29.187386 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:29.187286    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 21:47:30.186839 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:30.186119    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 21:47:30.186839 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:30.186196    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 21:47:30.457864 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:30.457779    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:30.504035 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:30.503992    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"]
Nov 03 21:47:30.594730 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:30.594696    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:30.713668 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:30.713568    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:31.192562 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:31.192499    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 21:47:31.193213 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:31.193183    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 21:47:32.186071 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:32.186026    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 21:47:32.186258 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:32.186097    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 21:47:32.588241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:32.588137    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:33.185960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:33.185914    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 21:47:33.186167 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:33.186001    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 21:47:34.146523 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:34.146481    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:34.186170 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:34.186125    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 21:47:34.186482 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:34.186451    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 21:47:34.802160 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:34.802125    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:35.185718 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:35.185678    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 21:47:35.186263 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:35.185762    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 21:47:35.505389 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:35.505273    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v" podUID="68b43a4e-3607-4620-b7a6-baf9c35cbf15" containerName="payload-pre-processing" containerID="cri-o://42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42" gracePeriod=25
Nov 03 21:47:35.835053 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:35.835026    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"
Nov 03 21:47:35.880067 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:35.879240    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68b43a4e-3607-4620-b7a6-baf9c35cbf15-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68b43a4e-3607-4620-b7a6-baf9c35cbf15-plugins-config-volume\") pod \"68b43a4e-3607-4620-b7a6-baf9c35cbf15\" (UID: \"68b43a4e-3607-4620-b7a6-baf9c35cbf15\") "
Nov 03 21:47:35.880067 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:35.879301    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68b43a4e-3607-4620-b7a6-baf9c35cbf15-kube-api-access-8spsk\" (UniqueName: \"kubernetes.io/projected/68b43a4e-3607-4620-b7a6-baf9c35cbf15-kube-api-access-8spsk\") pod \"68b43a4e-3607-4620-b7a6-baf9c35cbf15\" (UID: \"68b43a4e-3607-4620-b7a6-baf9c35cbf15\") "
Nov 03 21:47:35.880067 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:35.879788    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68b43a4e-3607-4620-b7a6-baf9c35cbf15-plugins-config-volume" pod "68b43a4e-3607-4620-b7a6-baf9c35cbf15" (UID: "68b43a4e-3607-4620-b7a6-baf9c35cbf15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:47:35.880067 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:35.880030    4454 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68b43a4e-3607-4620-b7a6-baf9c35cbf15-plugins-config-volume\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:35.884129 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:35.884096    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68b43a4e-3607-4620-b7a6-baf9c35cbf15-kube-api-access-8spsk" pod "68b43a4e-3607-4620-b7a6-baf9c35cbf15" (UID: "68b43a4e-3607-4620-b7a6-baf9c35cbf15"). InnerVolumeSpecName "kube-api-access-8spsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:47:35.981339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:35.981295    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8spsk\" (UniqueName: \"kubernetes.io/projected/68b43a4e-3607-4620-b7a6-baf9c35cbf15-kube-api-access-8spsk\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:36.150228 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.150187    4454 generic.go:356] "Generic (PLEG): container finished" podID="68b43a4e-3607-4620-b7a6-baf9c35cbf15" containerID="42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42" exitCode=0
Nov 03 21:47:36.150228 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.150230    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"
Nov 03 21:47:36.150436 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.150272    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v" event={"ID":"68b43a4e-3607-4620-b7a6-baf9c35cbf15","Type":"ContainerDied","Data":"42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42"}
Nov 03 21:47:36.150436 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.150326    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v" event={"ID":"68b43a4e-3607-4620-b7a6-baf9c35cbf15","Type":"ContainerDied","Data":"6e6abbcaf6d11d1c4c963a789e0a42413463e5729ea51f957423e7eceede84ef"}
Nov 03 21:47:36.150436 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.150356    4454 scope.go:122] "RemoveContainer" containerID="42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42"
Nov 03 21:47:36.160639 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.160620    4454 scope.go:122] "RemoveContainer" containerID="42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42"
Nov 03 21:47:36.160912 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:47:36.160894    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42\": container with ID starting with 42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42 not found: ID does not exist" containerID="42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42"
Nov 03 21:47:36.160972 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.160924    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42"} err="failed to get container status \"42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42\": rpc error: code = NotFound desc = could not find container \"42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42\": container with ID starting with 42021ce90a1b48c65069db3b8267083666ecd4d7e54ae84112f63407021fee42 not found: ID does not exist"
Nov 03 21:47:36.189492 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.189471    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:36.189826 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.189773    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:36.190474 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.190457    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:47:36.198821 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.198800    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"]
Nov 03 21:47:36.202246 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.202228    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-dd5620-577d4849cf-s6j5v"]
Nov 03 21:47:36.713197 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:36.713161    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:37.232035 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:37.232004    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68b43a4e-3607-4620-b7a6-baf9c35cbf15" path="/var/lib/kubelet/pods/68b43a4e-3607-4620-b7a6-baf9c35cbf15/volumes"
Nov 03 21:47:37.558575 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:37.558482    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:39.111234 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:39.111195    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"]
Nov 03 21:47:39.122285 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:39.122258    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"]
Nov 03 21:47:39.215468 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:39.215423    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:39.824385 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:39.824349    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:41.672830 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:41.672796    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:43.352054 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:43.352020    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:43.677851 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:43.677768    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:44.986204 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:44.986167    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:45.118543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:45.118488    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:45.399395 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:45.399362    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:45.703885 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:45.703797    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:47.668565 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:47.668524    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:47.886031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:47.885990    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:47.986724 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:47.986639    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:48.648794 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:48.648754    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"]
Nov 03 21:47:48.744102 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:48.744064    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:48.967620 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:48.967541    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:49.230591 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:49.230452    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:50.384280 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:50.384246    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:50.491793 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:50.491758    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"]
Nov 03 21:47:50.495305 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:50.495251    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:50.510564 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:50.510531    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"]
Nov 03 21:47:50.510731 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:50.510591    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"]
Nov 03 21:47:50.590612 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:50.590522    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:51.099574 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:51.099532    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:52.372988 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:47:52.372930    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:47:52.591745 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:52.591709    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:53.940467 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:53.940435    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"]
Nov 03 21:47:54.040316 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.040273    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:54.133834 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.133788    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" containerID="cri-o://59f0e5bdf5aff05f6e56cdf7f83355d16c440e5aabc8b7ccf76465d5c1ef7503" gracePeriod=45
Nov 03 21:47:54.143132 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.143097    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main" containerID="cri-o://1639e42dd36b2016482a52500c1fa7d0255ecd3d2c76e2db3e04c830d61efbc5" gracePeriod=45
Nov 03 21:47:54.275179 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.275144    4454 generic.go:356] "Generic (PLEG): container finished" podID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerID="1639e42dd36b2016482a52500c1fa7d0255ecd3d2c76e2db3e04c830d61efbc5" exitCode=0
Nov 03 21:47:54.275365 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.275214    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" event={"ID":"6450e3ec-6e14-4520-94e9-3d41d5b578f8","Type":"ContainerDied","Data":"1639e42dd36b2016482a52500c1fa7d0255ecd3d2c76e2db3e04c830d61efbc5"}
Nov 03 21:47:54.277000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.276976    4454 generic.go:356] "Generic (PLEG): container finished" podID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerID="59f0e5bdf5aff05f6e56cdf7f83355d16c440e5aabc8b7ccf76465d5c1ef7503" exitCode=0
Nov 03 21:47:54.277141 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.277025    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" event={"ID":"f09c8d95-070e-4706-8ddd-92ddaea9c8d8","Type":"ContainerDied","Data":"59f0e5bdf5aff05f6e56cdf7f83355d16c440e5aabc8b7ccf76465d5c1ef7503"}
Nov 03 21:47:54.340844 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.340618    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": dial tcp 10.133.0.58:8000: connect: connection refused"
Nov 03 21:47:54.428654 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.428489    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:54.461028 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.460999    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:47:54.468295 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.468230    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:47:54.555945 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.555886    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:54.612384 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.612352    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tls-certs\" (UniqueName: \"kubernetes.io/secret/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tls-certs\") pod \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") "
Nov 03 21:47:54.612634 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.612613    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tls-certs\" (UniqueName: \"kubernetes.io/secret/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tls-certs\") pod \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") "
Nov 03 21:47:54.612721 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.612674    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-model-cache\") pod \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") "
Nov 03 21:47:54.612777 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.612728    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6450e3ec-6e14-4520-94e9-3d41d5b578f8-kube-api-access-99cc6\" (UniqueName: \"kubernetes.io/projected/6450e3ec-6e14-4520-94e9-3d41d5b578f8-kube-api-access-99cc6\") pod \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") "
Nov 03 21:47:54.612828 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.612795    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-kube-api-access-s2s2j\" (UniqueName: \"kubernetes.io/projected/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-kube-api-access-s2s2j\") pod \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") "
Nov 03 21:47:54.612881 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.612849    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-home\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-home\") pod \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") "
Nov 03 21:47:54.612937 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.612879    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tmp-dir\") pod \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") "
Nov 03 21:47:54.612937 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.612925    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-dshm\") pod \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") "
Nov 03 21:47:54.613034 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.612959    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-home\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-home\") pod \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") "
Nov 03 21:47:54.613034 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.612993    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tmp-dir\") pod \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") "
Nov 03 21:47:54.613128 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.613047    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-model-cache\") pod \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\" (UID: \"6450e3ec-6e14-4520-94e9-3d41d5b578f8\") "
Nov 03 21:47:54.613179 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.613144    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-dshm\") pod \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\" (UID: \"f09c8d95-070e-4706-8ddd-92ddaea9c8d8\") "
Nov 03 21:47:54.613395 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.612955    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-model-cache" pod "f09c8d95-070e-4706-8ddd-92ddaea9c8d8" (UID: "f09c8d95-070e-4706-8ddd-92ddaea9c8d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:47:54.613632 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.613607    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-model-cache" pod "6450e3ec-6e14-4520-94e9-3d41d5b578f8" (UID: "6450e3ec-6e14-4520-94e9-3d41d5b578f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:47:54.613719 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.613164    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-home" pod "6450e3ec-6e14-4520-94e9-3d41d5b578f8" (UID: "6450e3ec-6e14-4520-94e9-3d41d5b578f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:47:54.613719 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.613214    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tmp-dir" pod "6450e3ec-6e14-4520-94e9-3d41d5b578f8" (UID: "6450e3ec-6e14-4520-94e9-3d41d5b578f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:47:54.613719 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.613367    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tmp-dir" pod "f09c8d95-070e-4706-8ddd-92ddaea9c8d8" (UID: "f09c8d95-070e-4706-8ddd-92ddaea9c8d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:47:54.613719 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.613405    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-home" pod "f09c8d95-070e-4706-8ddd-92ddaea9c8d8" (UID: "f09c8d95-070e-4706-8ddd-92ddaea9c8d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:47:54.614240 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.614221    4454 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-model-cache\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:54.614240 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.614240    4454 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-home\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:54.614377 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.614249    4454 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tmp-dir\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:54.614377 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.614257    4454 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-home\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:54.614377 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.614264    4454 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tmp-dir\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:54.614377 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.614272    4454 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-model-cache\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:54.614919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.614893    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tls-certs" pod "f09c8d95-070e-4706-8ddd-92ddaea9c8d8" (UID: "f09c8d95-070e-4706-8ddd-92ddaea9c8d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:47:54.615353 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.615333    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6450e3ec-6e14-4520-94e9-3d41d5b578f8-kube-api-access-99cc6" pod "6450e3ec-6e14-4520-94e9-3d41d5b578f8" (UID: "6450e3ec-6e14-4520-94e9-3d41d5b578f8"). InnerVolumeSpecName "kube-api-access-99cc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:47:54.615724 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.615695    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tls-certs" pod "6450e3ec-6e14-4520-94e9-3d41d5b578f8" (UID: "6450e3ec-6e14-4520-94e9-3d41d5b578f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:47:54.615806 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.615776    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-kube-api-access-s2s2j" pod "f09c8d95-070e-4706-8ddd-92ddaea9c8d8" (UID: "f09c8d95-070e-4706-8ddd-92ddaea9c8d8"). InnerVolumeSpecName "kube-api-access-s2s2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:47:54.615806 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.615773    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-dshm" pod "6450e3ec-6e14-4520-94e9-3d41d5b578f8" (UID: "6450e3ec-6e14-4520-94e9-3d41d5b578f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:47:54.616440 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.616422    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-dshm" pod "f09c8d95-070e-4706-8ddd-92ddaea9c8d8" (UID: "f09c8d95-070e-4706-8ddd-92ddaea9c8d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:47:54.716194 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.716161    4454 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-tls-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:54.716194 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.716197    4454 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6450e3ec-6e14-4520-94e9-3d41d5b578f8-tls-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:54.716384 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.716212    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99cc6\" (UniqueName: \"kubernetes.io/projected/6450e3ec-6e14-4520-94e9-3d41d5b578f8-kube-api-access-99cc6\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:54.716384 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.716227    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2s2j\" (UniqueName: \"kubernetes.io/projected/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-kube-api-access-s2s2j\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:54.716384 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.716241    4454 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6450e3ec-6e14-4520-94e9-3d41d5b578f8-dshm\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:54.716384 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:54.716257    4454 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f09c8d95-070e-4706-8ddd-92ddaea9c8d8-dshm\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:47:55.284281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:55.284252    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"
Nov 03 21:47:55.284281 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:55.284264    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv" event={"ID":"6450e3ec-6e14-4520-94e9-3d41d5b578f8","Type":"ContainerDied","Data":"4dbc8fc3fa60b0fccd604c398baca904276667de33b5f0015f4218da53ae7902"}
Nov 03 21:47:55.284875 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:55.284319    4454 scope.go:122] "RemoveContainer" containerID="1639e42dd36b2016482a52500c1fa7d0255ecd3d2c76e2db3e04c830d61efbc5"
Nov 03 21:47:55.285821 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:55.285795    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q" event={"ID":"f09c8d95-070e-4706-8ddd-92ddaea9c8d8","Type":"ContainerDied","Data":"b50e8e5073416646a53d35a95ef98b08ef2f08954464d7b0f38b466755544885"}
Nov 03 21:47:55.285906 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:55.285886    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"
Nov 03 21:47:55.296233 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:55.296206    4454 scope.go:122] "RemoveContainer" containerID="59f0e5bdf5aff05f6e56cdf7f83355d16c440e5aabc8b7ccf76465d5c1ef7503"
Nov 03 21:47:55.333280 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:55.333253    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"]
Nov 03 21:47:55.342061 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:55.342032    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-e246f9/distinct-w0-e246f9-kserve-5d9689d6c7-vj57q"]
Nov 03 21:47:55.375333 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:55.375296    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"]
Nov 03 21:47:55.380407 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:55.380372    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-e246f9/premium-w0-e246f9-kserve-f5bf57d4b-c56kv"]
Nov 03 21:47:56.651532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:56.636931    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"]
Nov 03 21:47:56.683707 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:56.675026    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:56.984876 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:56.983781    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:57.458537 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:57.453427    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:57.463678 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:57.455380    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" path="/var/lib/kubelet/pods/6450e3ec-6e14-4520-94e9-3d41d5b578f8/volumes"
Nov 03 21:47:57.468751 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:57.468723    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" path="/var/lib/kubelet/pods/f09c8d95-070e-4706-8ddd-92ddaea9c8d8/volumes"
Nov 03 21:47:57.573448 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:57.573415    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:57.768721 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:57.768597    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:57.802693 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:57.802645    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:57.911671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:57.911624    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:57.992540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:57.913377    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:47:57.992540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:57.952682    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:58.022657 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.022568    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"]
Nov 03 21:47:58.023142 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.023108    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w" podUID="c6c694f6-eeeb-4b68-89ca-64447fa9955d" containerName="istio-proxy" containerID="cri-o://10410980cf7cf028a2d1c434ac95d7b64807b6c85ac4088d50f7c464aa02cbd4" gracePeriod=30
Nov 03 21:47:58.044085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.042215    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"]
Nov 03 21:47:58.044085 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.042460    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx" podUID="28b54323-d6b0-40b7-ba3f-97ea85d99d18" containerName="istio-proxy" containerID="cri-o://9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b" gracePeriod=30
Nov 03 21:47:58.044711 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.044681    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:58.477530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.464592    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:58.750464 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.749707    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:58.780705 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.769384    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Nov 03 21:47:58.780705 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.769804    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:47:58.809687 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.809646    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": context deadline exceeded"
Nov 03 21:47:58.811374 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.810451    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:47:58.867175 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.867134    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:58.982926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.982876    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": context deadline exceeded"
Nov 03 21:47:59.004590 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:58.983533    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:59.005163 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.001615    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:59.005623 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.002048    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:47:59.005883 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.005867    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:47:59.006023 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.006012    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:47:59.654591 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.468600    4454 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": context deadline exceeded"
Nov 03 21:47:59.654591 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.468927    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Nov 03 21:47:59.779540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.750483    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:59.801891 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.798553    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:47:59.822392 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.813136    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:59.822392 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.816904    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:47:59.888552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.877328    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:47:59.888552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.877394    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:47:59.975866 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.975771    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Nov 03 21:48:00.005590 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:47:59.997330    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:48:00.033529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:00.008189    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:00.036462 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:00.034601    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:00.657535 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:00.657463    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 03 21:48:00.753557 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:00.752919    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 21:48:00.867535 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:00.848543    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:00.892838 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:00.892793    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Nov 03 21:48:00.962985 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:00.960948    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:00.979639 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:00.979595    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:01.036632 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.036584    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:48:01.191694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.191658    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"]
Nov 03 21:48:01.197524 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.197383    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": context deadline exceeded"
Nov 03 21:48:01.219620 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.218566    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:01.236210 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.229436    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:48:01.343082 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.338319    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:48:01.361228 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.361068    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:01.388773 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.362791    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:48:01.391099 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.391074    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:48:01.670026 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.658602    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 03 21:48:01.670026 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.662414    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f" podUID="b922f440-4072-4cfd-8cb1-f1a3646daaca" containerName="payload-pre-processing" containerID="cri-o://1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538" gracePeriod=25
Nov 03 21:48:01.700667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.700633    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:48:01.788588 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.781977    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:48:01.813487 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.813047    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:48:01.890159 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:01.890130    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:48:02.691278 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:02.691240    4454 patch_prober.go:29] interesting pod/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:48:02.691469 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:02.691315    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w" podUID="c6c694f6-eeeb-4b68-89ca-64447fa9955d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:48:03.318795 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.318625    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w" event={"ID":"c6c694f6-eeeb-4b68-89ca-64447fa9955d","Type":"ContainerDied","Data":"10410980cf7cf028a2d1c434ac95d7b64807b6c85ac4088d50f7c464aa02cbd4"}
Nov 03 21:48:03.318795 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.318679    4454 generic.go:356] "Generic (PLEG): container finished" podID="c6c694f6-eeeb-4b68-89ca-64447fa9955d" containerID="10410980cf7cf028a2d1c434ac95d7b64807b6c85ac4088d50f7c464aa02cbd4" exitCode=0
Nov 03 21:48:03.353524 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.353491    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:48:03.446368 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.446331    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-kube-api-access-cxdp2\" (UniqueName: \"kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-kube-api-access-cxdp2\") pod \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") "
Nov 03 21:48:03.446827 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.446628    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-credential-socket\") pod \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") "
Nov 03 21:48:03.446827 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.446670    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-token\" (UniqueName: \"kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-token\") pod \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") "
Nov 03 21:48:03.446827 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.446745    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-certs\") pod \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") "
Nov 03 21:48:03.446979 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.446865    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istiod-ca-cert\") pod \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") "
Nov 03 21:48:03.446979 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.446915    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-data\") pod \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") "
Nov 03 21:48:03.446979 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.446949    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-socket\") pod \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") "
Nov 03 21:48:03.447129 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.447006    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-podinfo\") pod \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") "
Nov 03 21:48:03.447129 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.447037    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-envoy\") pod \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\" (UID: \"c6c694f6-eeeb-4b68-89ca-64447fa9955d\") "
Nov 03 21:48:03.449735 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.448072    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-certs" pod "c6c694f6-eeeb-4b68-89ca-64447fa9955d" (UID: "c6c694f6-eeeb-4b68-89ca-64447fa9955d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:03.449735 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.448221    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-data" pod "c6c694f6-eeeb-4b68-89ca-64447fa9955d" (UID: "c6c694f6-eeeb-4b68-89ca-64447fa9955d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:03.449735 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.448598    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-socket" pod "c6c694f6-eeeb-4b68-89ca-64447fa9955d" (UID: "c6c694f6-eeeb-4b68-89ca-64447fa9955d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:03.449735 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.448715    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istiod-ca-cert" pod "c6c694f6-eeeb-4b68-89ca-64447fa9955d" (UID: "c6c694f6-eeeb-4b68-89ca-64447fa9955d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:03.449735 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.449316    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-credential-socket" pod "c6c694f6-eeeb-4b68-89ca-64447fa9955d" (UID: "c6c694f6-eeeb-4b68-89ca-64447fa9955d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:03.450993 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.450886    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-envoy" pod "c6c694f6-eeeb-4b68-89ca-64447fa9955d" (UID: "c6c694f6-eeeb-4b68-89ca-64447fa9955d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:03.451195 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.451169    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-kube-api-access-cxdp2" pod "c6c694f6-eeeb-4b68-89ca-64447fa9955d" (UID: "c6c694f6-eeeb-4b68-89ca-64447fa9955d"). InnerVolumeSpecName "kube-api-access-cxdp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:03.455183 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.455160    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-podinfo" pod "c6c694f6-eeeb-4b68-89ca-64447fa9955d" (UID: "c6c694f6-eeeb-4b68-89ca-64447fa9955d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:03.458735 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.458712    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-token" pod "c6c694f6-eeeb-4b68-89ca-64447fa9955d" (UID: "c6c694f6-eeeb-4b68-89ca-64447fa9955d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:03.549518 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.549477    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:03.549518 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.549520    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:03.549706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.549534    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:03.549706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.549543    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:03.549706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.549551    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:03.549706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.549560    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:03.549706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.549570    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxdp2\" (UniqueName: \"kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-kube-api-access-cxdp2\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:03.549706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.549578    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6c694f6-eeeb-4b68-89ca-64447fa9955d-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:03.549706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.549587    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6c694f6-eeeb-4b68-89ca-64447fa9955d-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:03.670394 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.670351    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" containerID="cri-o://37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2" gracePeriod=45
Nov 03 21:48:03.698560 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.698527    4454 patch_prober.go:29] interesting pod/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:48:03.698679 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.698586    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx" podUID="28b54323-d6b0-40b7-ba3f-97ea85d99d18" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:48:03.834874 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.833387    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/ready\": dial tcp 10.133.0.56:8000: connect: connection refused"
Nov 03 21:48:03.969992 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:03.969967    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:48:04.059776 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.059745    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tls-certs\" (UniqueName: \"kubernetes.io/secret/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tls-certs\") pod \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") "
Nov 03 21:48:04.059976 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.059955    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-kube-api-access-q9fmw\" (UniqueName: \"kubernetes.io/projected/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-kube-api-access-q9fmw\") pod \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") "
Nov 03 21:48:04.060048 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.059997    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-dshm\") pod \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") "
Nov 03 21:48:04.060103 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.060048    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-home\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-home\") pod \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") "
Nov 03 21:48:04.060153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.060122    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tmp-dir\") pod \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") "
Nov 03 21:48:04.060284 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.060209    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-model-cache\") pod \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\" (UID: \"2007d8ff-83cc-42bc-9f9d-766b4ffda89b\") "
Nov 03 21:48:04.061444 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.061381    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-home" pod "2007d8ff-83cc-42bc-9f9d-766b4ffda89b" (UID: "2007d8ff-83cc-42bc-9f9d-766b4ffda89b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:04.063179 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.062962    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tmp-dir" pod "2007d8ff-83cc-42bc-9f9d-766b4ffda89b" (UID: "2007d8ff-83cc-42bc-9f9d-766b4ffda89b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:04.063179 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.063091    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-model-cache" pod "2007d8ff-83cc-42bc-9f9d-766b4ffda89b" (UID: "2007d8ff-83cc-42bc-9f9d-766b4ffda89b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:04.063803 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.063781    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-dshm" pod "2007d8ff-83cc-42bc-9f9d-766b4ffda89b" (UID: "2007d8ff-83cc-42bc-9f9d-766b4ffda89b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:04.064245 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.064221    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tls-certs" pod "2007d8ff-83cc-42bc-9f9d-766b4ffda89b" (UID: "2007d8ff-83cc-42bc-9f9d-766b4ffda89b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:04.064568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.064544    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-kube-api-access-q9fmw" pod "2007d8ff-83cc-42bc-9f9d-766b4ffda89b" (UID: "2007d8ff-83cc-42bc-9f9d-766b4ffda89b"). InnerVolumeSpecName "kube-api-access-q9fmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:04.163185 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.163151    4454 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tls-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:04.163185 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.163188    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9fmw\" (UniqueName: \"kubernetes.io/projected/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-kube-api-access-q9fmw\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:04.163185 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.163202    4454 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-dshm\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:04.163500 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.163215    4454 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-home\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:04.163500 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.163226    4454 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-tmp-dir\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:04.163500 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.163240    4454 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2007d8ff-83cc-42bc-9f9d-766b4ffda89b-model-cache\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:04.327121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.327018    4454 generic.go:356] "Generic (PLEG): container finished" podID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerID="37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2" exitCode=0
Nov 03 21:48:04.327121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.327076    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"
Nov 03 21:48:04.327669 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.327150    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" event={"ID":"2007d8ff-83cc-42bc-9f9d-766b4ffda89b","Type":"ContainerDied","Data":"37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2"}
Nov 03 21:48:04.327669 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.327198    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj" event={"ID":"2007d8ff-83cc-42bc-9f9d-766b4ffda89b","Type":"ContainerDied","Data":"5f4023e8e56578f688c23124d17ce1b23487d79e978c40f02a2e556abaa56dd8"}
Nov 03 21:48:04.327669 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.327223    4454 scope.go:122] "RemoveContainer" containerID="37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2"
Nov 03 21:48:04.328770 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.328747    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w" event={"ID":"c6c694f6-eeeb-4b68-89ca-64447fa9955d","Type":"ContainerDied","Data":"d7d6bab45a60f20bfafe0999265c4b4a9946aeb6f64ee49bfeef266f7a92b15b"}
Nov 03 21:48:04.328891 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.328867    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"
Nov 03 21:48:04.339096 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.339077    4454 scope.go:122] "RemoveContainer" containerID="37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2"
Nov 03 21:48:04.339413 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:04.339376    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2\": container with ID starting with 37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2 not found: ID does not exist" containerID="37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2"
Nov 03 21:48:04.339562 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.339408    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2"} err="failed to get container status \"37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2\": rpc error: code = NotFound desc = could not find container \"37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2\": container with ID starting with 37ed8a00d11ffff21334bf29202c26b6ea6c01ce6debcfdcc18acc5a7701e7a2 not found: ID does not exist"
Nov 03 21:48:04.339562 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.339441    4454 scope.go:122] "RemoveContainer" containerID="10410980cf7cf028a2d1c434ac95d7b64807b6c85ac4088d50f7c464aa02cbd4"
Nov 03 21:48:04.375815 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.375782    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"]
Nov 03 21:48:04.378836 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.378818    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-0a6da1/distinct-w5-0a6da1-kserve-7458dc659f-x6mfj"]
Nov 03 21:48:04.420609 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.420577    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"]
Nov 03 21:48:04.423595 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:04.423566    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-x265w"]
Nov 03 21:48:05.231482 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:05.231444    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" path="/var/lib/kubelet/pods/2007d8ff-83cc-42bc-9f9d-766b4ffda89b/volumes"
Nov 03 21:48:05.232074 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:05.232052    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6c694f6-eeeb-4b68-89ca-64447fa9955d" path="/var/lib/kubelet/pods/c6c694f6-eeeb-4b68-89ca-64447fa9955d/volumes"
Nov 03 21:48:05.452061 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:05.452029    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:05.524938 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:05.524696    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" containerID="cri-o://77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5" gracePeriod=45
Nov 03 21:48:05.549578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:05.547446    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" containerID="cri-o://5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1" gracePeriod=45
Nov 03 21:48:05.549578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:05.547667    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main" containerID="cri-o://bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128" gracePeriod=45
Nov 03 21:48:05.739567 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:05.739331    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": dial tcp 10.133.0.50:8000: connect: connection refused"
Nov 03 21:48:05.739786 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:05.739690    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": dial tcp 10.133.0.66:8000: connect: connection refused"
Nov 03 21:48:05.939931 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:05.939681    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:48:05.942775 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:05.942751    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:48:05.946983 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:05.946962    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:48:06.046662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.046639    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:48:06.093958 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.093916    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2700b70-1f64-4db1-b6c2-552669059d93-kube-api-access-hmbjw\" (UniqueName: \"kubernetes.io/projected/a2700b70-1f64-4db1-b6c2-552669059d93-kube-api-access-hmbjw\") pod \"a2700b70-1f64-4db1-b6c2-552669059d93\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") "
Nov 03 21:48:06.094136 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.093975    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-certs\") pod \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") "
Nov 03 21:48:06.094136 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.094045    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-dshm\") pod \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") "
Nov 03 21:48:06.094136 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.094081    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-tmp-dir\") pod \"a2700b70-1f64-4db1-b6c2-552669059d93\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") "
Nov 03 21:48:06.094136 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.094121    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-token\" (UniqueName: \"kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-token\") pod \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") "
Nov 03 21:48:06.094354 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.094162    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2700b70-1f64-4db1-b6c2-552669059d93-tls-certs\" (UniqueName: \"kubernetes.io/secret/a2700b70-1f64-4db1-b6c2-552669059d93-tls-certs\") pod \"a2700b70-1f64-4db1-b6c2-552669059d93\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") "
Nov 03 21:48:06.094354 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.094200    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tls-certs\" (UniqueName: \"kubernetes.io/secret/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tls-certs\") pod \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") "
Nov 03 21:48:06.094354 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.094311    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tls-certs\" (UniqueName: \"kubernetes.io/secret/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tls-certs\") pod \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") "
Nov 03 21:48:06.094354 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.094342    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tmp-dir\") pod \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098290    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-dshm\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-dshm\") pod \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098369    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-dshm\") pod \"a2700b70-1f64-4db1-b6c2-552669059d93\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098421    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-kube-api-access-x44r6\" (UniqueName: \"kubernetes.io/projected/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-kube-api-access-x44r6\") pod \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098449    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-model-cache\") pod \"a2700b70-1f64-4db1-b6c2-552669059d93\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098496    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istiod-ca-cert\") pod \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098552    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-home\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-home\") pod \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098581    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-kube-api-access-v2cq8\" (UniqueName: \"kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-kube-api-access-v2cq8\") pod \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098654    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-model-cache\") pod \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098694    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/063c1a56-0889-40ac-8eca-3f7acb93f7cb-kube-api-access-sfv8n\" (UniqueName: \"kubernetes.io/projected/063c1a56-0889-40ac-8eca-3f7acb93f7cb-kube-api-access-sfv8n\") pod \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098720    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-home\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-home\") pod \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\" (UID: \"063c1a56-0889-40ac-8eca-3f7acb93f7cb\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098766    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-socket\") pod \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098812    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-credential-socket\") pod \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098874    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tmp-dir\") pod \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098904    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-data\") pod \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.098955    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-model-cache\") pod \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\" (UID: \"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.099006    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-podinfo\") pod \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.099061    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-envoy\") pod \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\" (UID: \"28b54323-d6b0-40b7-ba3f-97ea85d99d18\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.099122    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-home\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-home\") pod \"a2700b70-1f64-4db1-b6c2-552669059d93\" (UID: \"a2700b70-1f64-4db1-b6c2-552669059d93\") "
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.100115    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-home" pod "a2700b70-1f64-4db1-b6c2-552669059d93" (UID: "a2700b70-1f64-4db1-b6c2-552669059d93"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.100770    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istiod-ca-cert" pod "28b54323-d6b0-40b7-ba3f-97ea85d99d18" (UID: "28b54323-d6b0-40b7-ba3f-97ea85d99d18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.101029    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-certs" pod "28b54323-d6b0-40b7-ba3f-97ea85d99d18" (UID: "28b54323-d6b0-40b7-ba3f-97ea85d99d18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.103529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.103055    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tmp-dir" pod "063c1a56-0889-40ac-8eca-3f7acb93f7cb" (UID: "063c1a56-0889-40ac-8eca-3f7acb93f7cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.104468 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.104260    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-model-cache" pod "a2700b70-1f64-4db1-b6c2-552669059d93" (UID: "a2700b70-1f64-4db1-b6c2-552669059d93"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.107290 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.107260    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tls-certs" pod "063c1a56-0889-40ac-8eca-3f7acb93f7cb" (UID: "063c1a56-0889-40ac-8eca-3f7acb93f7cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:06.107501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.107479    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-socket" pod "28b54323-d6b0-40b7-ba3f-97ea85d99d18" (UID: "28b54323-d6b0-40b7-ba3f-97ea85d99d18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.107705 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.107686    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-home" pod "300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" (UID: "300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.108376 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.108348    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-tmp-dir" pod "a2700b70-1f64-4db1-b6c2-552669059d93" (UID: "a2700b70-1f64-4db1-b6c2-552669059d93"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.108645 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.108621    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-credential-socket" pod "28b54323-d6b0-40b7-ba3f-97ea85d99d18" (UID: "28b54323-d6b0-40b7-ba3f-97ea85d99d18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.108831 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.108810    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tmp-dir" pod "300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" (UID: "300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.109000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.108980    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-data" pod "28b54323-d6b0-40b7-ba3f-97ea85d99d18" (UID: "28b54323-d6b0-40b7-ba3f-97ea85d99d18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.109248 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.109224    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-model-cache" pod "300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" (UID: "300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.110433 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.110254    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-kube-api-access-v2cq8" pod "28b54323-d6b0-40b7-ba3f-97ea85d99d18" (UID: "28b54323-d6b0-40b7-ba3f-97ea85d99d18"). InnerVolumeSpecName "kube-api-access-v2cq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:06.110862 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.110630    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-home" pod "063c1a56-0889-40ac-8eca-3f7acb93f7cb" (UID: "063c1a56-0889-40ac-8eca-3f7acb93f7cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.110862 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.110695    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-dshm" pod "063c1a56-0889-40ac-8eca-3f7acb93f7cb" (UID: "063c1a56-0889-40ac-8eca-3f7acb93f7cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.111275 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.111045    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-model-cache" pod "063c1a56-0889-40ac-8eca-3f7acb93f7cb" (UID: "063c1a56-0889-40ac-8eca-3f7acb93f7cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.113086 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.113060    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-podinfo" pod "28b54323-d6b0-40b7-ba3f-97ea85d99d18" (UID: "28b54323-d6b0-40b7-ba3f-97ea85d99d18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:06.113209 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.113184    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2700b70-1f64-4db1-b6c2-552669059d93-kube-api-access-hmbjw" pod "a2700b70-1f64-4db1-b6c2-552669059d93" (UID: "a2700b70-1f64-4db1-b6c2-552669059d93"). InnerVolumeSpecName "kube-api-access-hmbjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:06.113395 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.113372    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-token" pod "28b54323-d6b0-40b7-ba3f-97ea85d99d18" (UID: "28b54323-d6b0-40b7-ba3f-97ea85d99d18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:06.113813 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.113493    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-dshm" pod "a2700b70-1f64-4db1-b6c2-552669059d93" (UID: "a2700b70-1f64-4db1-b6c2-552669059d93"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.113813 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.113780    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-envoy" pod "28b54323-d6b0-40b7-ba3f-97ea85d99d18" (UID: "28b54323-d6b0-40b7-ba3f-97ea85d99d18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.114414 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.114388    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2700b70-1f64-4db1-b6c2-552669059d93-tls-certs" pod "a2700b70-1f64-4db1-b6c2-552669059d93" (UID: "a2700b70-1f64-4db1-b6c2-552669059d93"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:06.114672 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.114651    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tls-certs" pod "300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" (UID: "300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:06.114859 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.114830    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/063c1a56-0889-40ac-8eca-3f7acb93f7cb-kube-api-access-sfv8n" pod "063c1a56-0889-40ac-8eca-3f7acb93f7cb" (UID: "063c1a56-0889-40ac-8eca-3f7acb93f7cb"). InnerVolumeSpecName "kube-api-access-sfv8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:06.115144 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.115120    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-dshm" pod "300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" (UID: "300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:06.115367 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.115348    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-kube-api-access-x44r6" pod "300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" (UID: "300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac"). InnerVolumeSpecName "kube-api-access-x44r6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:06.178850 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.178806    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:06.186678 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.186646    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f" podUID="b922f440-4072-4cfd-8cb1-f1a3646daaca" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:48:06.192330 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.192301    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2" podUID="33553b17-5b86-4d27-91d7-1d0d29a75280" containerName="payload-pre-processing" containerID="cri-o://8a24fabef0f083d5e7809d8499b0c1eb3c055247a8133ee740af6d18f3b949e5" gracePeriod=25
Nov 03 21:48:06.201398 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201357    4454 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-model-cache\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201398 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201392    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfv8n\" (UniqueName: \"kubernetes.io/projected/063c1a56-0889-40ac-8eca-3f7acb93f7cb-kube-api-access-sfv8n\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201409    4454 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-home\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201419    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201433    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201443    4454 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tmp-dir\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201455    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201466    4454 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-model-cache\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201473    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201481    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201489    4454 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-home\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201499    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmbjw\" (UniqueName: \"kubernetes.io/projected/a2700b70-1f64-4db1-b6c2-552669059d93-kube-api-access-hmbjw\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201526    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28b54323-d6b0-40b7-ba3f-97ea85d99d18-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201538    4454 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-dshm\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201549    4454 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-tmp-dir\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201560    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.201572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201571    4454 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2700b70-1f64-4db1-b6c2-552669059d93-tls-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.202033 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201583    4454 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tls-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.202033 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201595    4454 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-tls-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.202033 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201607    4454 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/063c1a56-0889-40ac-8eca-3f7acb93f7cb-tmp-dir\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.202033 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201619    4454 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-dshm\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.202033 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201631    4454 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-dshm\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.202033 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201639    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x44r6\" (UniqueName: \"kubernetes.io/projected/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-kube-api-access-x44r6\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.202033 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201646    4454 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2700b70-1f64-4db1-b6c2-552669059d93-model-cache\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.202033 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201654    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28b54323-d6b0-40b7-ba3f-97ea85d99d18-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.202033 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201661    4454 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac-home\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.202033 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.201670    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2cq8\" (UniqueName: \"kubernetes.io/projected/28b54323-d6b0-40b7-ba3f-97ea85d99d18-kube-api-access-v2cq8\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.249767 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.249735    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"]
Nov 03 21:48:06.345227 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.345192    4454 generic.go:356] "Generic (PLEG): container finished" podID="a2700b70-1f64-4db1-b6c2-552669059d93" containerID="bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128" exitCode=0
Nov 03 21:48:06.345428 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.345260    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"
Nov 03 21:48:06.345428 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.345265    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" event={"ID":"a2700b70-1f64-4db1-b6c2-552669059d93","Type":"ContainerDied","Data":"bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128"}
Nov 03 21:48:06.345428 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.345315    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd" event={"ID":"a2700b70-1f64-4db1-b6c2-552669059d93","Type":"ContainerDied","Data":"2143d679ed21daae8892dfb3fa9d433004cb3e51c029875352226873418de37d"}
Nov 03 21:48:06.345428 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.345339    4454 scope.go:122] "RemoveContainer" containerID="bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128"
Nov 03 21:48:06.348173 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.347051    4454 generic.go:356] "Generic (PLEG): container finished" podID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerID="5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1" exitCode=0
Nov 03 21:48:06.348173 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.347100    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"
Nov 03 21:48:06.348173 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.347146    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" event={"ID":"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac","Type":"ContainerDied","Data":"5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1"}
Nov 03 21:48:06.348173 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.347181    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk" event={"ID":"300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac","Type":"ContainerDied","Data":"6b8ae227f700bb852292a572473b761fc6b870552f2ce56f3c1982e7c896d3df"}
Nov 03 21:48:06.351814 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.351793    4454 generic.go:356] "Generic (PLEG): container finished" podID="33553b17-5b86-4d27-91d7-1d0d29a75280" containerID="8a24fabef0f083d5e7809d8499b0c1eb3c055247a8133ee740af6d18f3b949e5" exitCode=0
Nov 03 21:48:06.352011 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.351860    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2" event={"ID":"33553b17-5b86-4d27-91d7-1d0d29a75280","Type":"ContainerDied","Data":"8a24fabef0f083d5e7809d8499b0c1eb3c055247a8133ee740af6d18f3b949e5"}
Nov 03 21:48:06.358707 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.358686    4454 generic.go:356] "Generic (PLEG): container finished" podID="28b54323-d6b0-40b7-ba3f-97ea85d99d18" containerID="9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b" exitCode=0
Nov 03 21:48:06.358866 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.358853    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"
Nov 03 21:48:06.359046 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.359024    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx" event={"ID":"28b54323-d6b0-40b7-ba3f-97ea85d99d18","Type":"ContainerDied","Data":"9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b"}
Nov 03 21:48:06.359422 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.359077    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx" event={"ID":"28b54323-d6b0-40b7-ba3f-97ea85d99d18","Type":"ContainerDied","Data":"8439e4b8388964794519899b7cee743e9c77cd31df1115da52e0bc30e4185797"}
Nov 03 21:48:06.361889 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.361860    4454 scope.go:122] "RemoveContainer" containerID="bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128"
Nov 03 21:48:06.362409 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:06.362163    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128\": container with ID starting with bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128 not found: ID does not exist" containerID="bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128"
Nov 03 21:48:06.362409 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.362195    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128"} err="failed to get container status \"bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128\": rpc error: code = NotFound desc = could not find container \"bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128\": container with ID starting with bc5e6077e8c78a12ab4e1f1204361da985f81f8c8be7961512b240233f211128 not found: ID does not exist"
Nov 03 21:48:06.362409 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.362216    4454 scope.go:122] "RemoveContainer" containerID="5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1"
Nov 03 21:48:06.397969 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.397534    4454 generic.go:356] "Generic (PLEG): container finished" podID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerID="77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5" exitCode=0
Nov 03 21:48:06.397969 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.397599    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" event={"ID":"063c1a56-0889-40ac-8eca-3f7acb93f7cb","Type":"ContainerDied","Data":"77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5"}
Nov 03 21:48:06.397969 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.397633    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655" event={"ID":"063c1a56-0889-40ac-8eca-3f7acb93f7cb","Type":"ContainerDied","Data":"a8d617baef90a968c09b81b3fb380beae4b5b1349f36f3c35034241146cd384a"}
Nov 03 21:48:06.399623 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.398601    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"
Nov 03 21:48:06.399623 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.399037    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:06.431132 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.431111    4454 scope.go:122] "RemoveContainer" containerID="5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1"
Nov 03 21:48:06.431640 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:06.431585    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1\": container with ID starting with 5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1 not found: ID does not exist" containerID="5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1"
Nov 03 21:48:06.431857 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.431813    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1"} err="failed to get container status \"5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1\": rpc error: code = NotFound desc = could not find container \"5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1\": container with ID starting with 5bf9cbebf264a98036bd3b08981e106facc2fa056fe1ee2e6965a142f7e9a3d1 not found: ID does not exist"
Nov 03 21:48:06.431971 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.431864    4454 scope.go:122] "RemoveContainer" containerID="9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b"
Nov 03 21:48:06.445357 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.445329    4454 scope.go:122] "RemoveContainer" containerID="9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b"
Nov 03 21:48:06.445781 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:06.445754    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b\": container with ID starting with 9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b not found: ID does not exist" containerID="9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b"
Nov 03 21:48:06.445885 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.445791    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b"} err="failed to get container status \"9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b\": rpc error: code = NotFound desc = could not find container \"9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b\": container with ID starting with 9a6dab26f934e068ade61b37747e11875109ddcbad25a0b1eeec59361c4cbc0b not found: ID does not exist"
Nov 03 21:48:06.445885 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.445827    4454 scope.go:122] "RemoveContainer" containerID="77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5"
Nov 03 21:48:06.462035 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.462017    4454 scope.go:122] "RemoveContainer" containerID="77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5"
Nov 03 21:48:06.462892 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:06.462859    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5\": container with ID starting with 77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5 not found: ID does not exist" containerID="77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5"
Nov 03 21:48:06.463152 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.462921    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5"} err="failed to get container status \"77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5\": rpc error: code = NotFound desc = could not find container \"77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5\": container with ID starting with 77914b0ecd0fd4741c8aeff1d746f09e3f8a1c9b270e4a3cbf872a9b483356e5 not found: ID does not exist"
Nov 03 21:48:06.465968 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.465913    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"]
Nov 03 21:48:06.469240 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.469213    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/premium-w1-4eced5-kserve-7bcb8d9c49-9pfxk"]
Nov 03 21:48:06.471031 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.471006    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"
Nov 03 21:48:06.496430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.496405    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"]
Nov 03 21:48:06.499959 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.499938    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/distinct-w1-4eced5-kserve-66d4d469fb-vxfbd"]
Nov 03 21:48:06.507244 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.507225    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33553b17-5b86-4d27-91d7-1d0d29a75280-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33553b17-5b86-4d27-91d7-1d0d29a75280-plugins-config-volume\") pod \"33553b17-5b86-4d27-91d7-1d0d29a75280\" (UID: \"33553b17-5b86-4d27-91d7-1d0d29a75280\") "
Nov 03 21:48:06.507331 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.507291    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33553b17-5b86-4d27-91d7-1d0d29a75280-kube-api-access-jqz2g\" (UniqueName: \"kubernetes.io/projected/33553b17-5b86-4d27-91d7-1d0d29a75280-kube-api-access-jqz2g\") pod \"33553b17-5b86-4d27-91d7-1d0d29a75280\" (UID: \"33553b17-5b86-4d27-91d7-1d0d29a75280\") "
Nov 03 21:48:06.507601 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.507583    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33553b17-5b86-4d27-91d7-1d0d29a75280-plugins-config-volume" pod "33553b17-5b86-4d27-91d7-1d0d29a75280" (UID: "33553b17-5b86-4d27-91d7-1d0d29a75280"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:06.507758 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.507745    4454 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33553b17-5b86-4d27-91d7-1d0d29a75280-plugins-config-volume\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:06.509253 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.509234    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33553b17-5b86-4d27-91d7-1d0d29a75280-kube-api-access-jqz2g" pod "33553b17-5b86-4d27-91d7-1d0d29a75280" (UID: "33553b17-5b86-4d27-91d7-1d0d29a75280"). InnerVolumeSpecName "kube-api-access-jqz2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:06.530473 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.530443    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"]
Nov 03 21:48:06.534189 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.534164    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd5620-openshift-default-86d5bc584f-5rfsx"]
Nov 03 21:48:06.566587 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.566546    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"]
Nov 03 21:48:06.569725 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.569702    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-4eced5/distinct-2-w1-4eced5-kserve-dbd886bb7-nq655"]
Nov 03 21:48:06.609029 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:06.609003    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqz2g\" (UniqueName: \"kubernetes.io/projected/33553b17-5b86-4d27-91d7-1d0d29a75280-kube-api-access-jqz2g\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:07.231560 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:07.231526    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" path="/var/lib/kubelet/pods/063c1a56-0889-40ac-8eca-3f7acb93f7cb/volumes"
Nov 03 21:48:07.232010 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:07.231996    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28b54323-d6b0-40b7-ba3f-97ea85d99d18" path="/var/lib/kubelet/pods/28b54323-d6b0-40b7-ba3f-97ea85d99d18/volumes"
Nov 03 21:48:07.232380 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:07.232368    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" path="/var/lib/kubelet/pods/300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac/volumes"
Nov 03 21:48:07.233096 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:07.233081    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" path="/var/lib/kubelet/pods/a2700b70-1f64-4db1-b6c2-552669059d93/volumes"
Nov 03 21:48:07.408112 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:07.408079    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2" event={"ID":"33553b17-5b86-4d27-91d7-1d0d29a75280","Type":"ContainerDied","Data":"55d267f3b8f8e152f7bb51c57c3931662474e4656314d2449f9510ba1ada9ed8"}
Nov 03 21:48:07.408112 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:07.408110    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"
Nov 03 21:48:07.408333 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:07.408137    4454 scope.go:122] "RemoveContainer" containerID="8a24fabef0f083d5e7809d8499b0c1eb3c055247a8133ee740af6d18f3b949e5"
Nov 03 21:48:07.450069 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:07.450028    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:07.456144 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:07.456120    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"]
Nov 03 21:48:07.466532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:07.466495    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-05bc27-687f5c8f6c-w95j2"]
Nov 03 21:48:08.962190 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:08.962149    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main" containerID="cri-o://79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6" gracePeriod=45
Nov 03 21:48:09.171838 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.171689    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:09.302847 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.301719    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33553b17-5b86-4d27-91d7-1d0d29a75280" path="/var/lib/kubelet/pods/33553b17-5b86-4d27-91d7-1d0d29a75280/volumes"
Nov 03 21:48:09.312556 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.307503    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:48:09.379848 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.379709    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-home\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-home\") pod \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") "
Nov 03 21:48:09.380053 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.380025    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-home" pod "4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" (UID: "4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:09.380559 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.380477    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tls-certs\") pod \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") "
Nov 03 21:48:09.380698 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.380584    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-model-cache\") pod \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") "
Nov 03 21:48:09.382255 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.380783    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-dshm\") pod \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") "
Nov 03 21:48:09.382255 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.380829    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-kube-api-access-bxskp\" (UniqueName: \"kubernetes.io/projected/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-kube-api-access-bxskp\") pod \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") "
Nov 03 21:48:09.382255 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.380899    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tmp-dir\") pod \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\" (UID: \"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9\") "
Nov 03 21:48:09.382255 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.381525    4454 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-home\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:09.382255 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.381740    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tmp-dir" pod "4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" (UID: "4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:09.382255 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.382219    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-model-cache" pod "4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" (UID: "4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:09.384629 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.384602    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-kube-api-access-bxskp" pod "4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" (UID: "4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9"). InnerVolumeSpecName "kube-api-access-bxskp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:09.388471 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.388447    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tls-certs" pod "4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" (UID: "4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:09.388727 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.388674    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-dshm" pod "4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" (UID: "4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:09.422874 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.422840    4454 generic.go:356] "Generic (PLEG): container finished" podID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerID="79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6" exitCode=0
Nov 03 21:48:09.423004 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.422992    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" event={"ID":"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9","Type":"ContainerDied","Data":"79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6"}
Nov 03 21:48:09.423044 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.423031    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82" event={"ID":"4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9","Type":"ContainerDied","Data":"aeb35e72dc374ae7e0e4c20cbc915d6f5990d49852c98583596a9e4af2038465"}
Nov 03 21:48:09.423075 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.423053    4454 scope.go:122] "RemoveContainer" containerID="79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6"
Nov 03 21:48:09.423176 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.423164    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"
Nov 03 21:48:09.445941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.445907    4454 scope.go:122] "RemoveContainer" containerID="79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6"
Nov 03 21:48:09.446402 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:09.446370    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6\": container with ID starting with 79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6 not found: ID does not exist" containerID="79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6"
Nov 03 21:48:09.446664 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.446633    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6"} err="failed to get container status \"79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6\": rpc error: code = NotFound desc = could not find container \"79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6\": container with ID starting with 79702143e9a5ba46739b4faca4a3024d90b77142f9c4e1b34923082eb8af56d6 not found: ID does not exist"
Nov 03 21:48:09.477617 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.477535    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"]
Nov 03 21:48:09.482644 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.482617    4454 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-dshm\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:09.482644 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.482647    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxskp\" (UniqueName: \"kubernetes.io/projected/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-kube-api-access-bxskp\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:09.482825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.482684    4454 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tmp-dir\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:09.482825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.482702    4454 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-tls-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:09.482825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.482715    4454 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9-model-cache\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:09.485644 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:09.485619    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-05bc27/distinct-w4-05bc27-kserve-79f7ff8454-k4r82"]
Nov 03 21:48:11.032975 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:11.032935    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:11.230414 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:11.230382    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" path="/var/lib/kubelet/pods/4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9/volumes"
Nov 03 21:48:12.760398 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:12.760361    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:13.729998 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:13.729958    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"]
Nov 03 21:48:13.753776 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:13.753744    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"]
Nov 03 21:48:13.846191 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:13.846156    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:13.945947 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:13.945910    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:14.557217 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:14.557186    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:16.185794 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:16.185754    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f" podUID="b922f440-4072-4cfd-8cb1-f1a3646daaca" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:48:16.371410 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:16.371378    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:16.471527 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:16.471434    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:16.775581 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:16.768283    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:17.842149 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:17.842112    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"]
Nov 03 21:48:17.842876 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:17.842841    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv" podUID="99af0d0b-bd91-4cc1-be41-d4414e335fac" containerName="maas-api" containerID="cri-o://62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9" gracePeriod=30
Nov 03 21:48:17.952755 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:17.952715    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:18.267355 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.265066    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:48:18.289547 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.286092    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99af0d0b-bd91-4cc1-be41-d4414e335fac-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99af0d0b-bd91-4cc1-be41-d4414e335fac-openshift-service-ca\") pod \"99af0d0b-bd91-4cc1-be41-d4414e335fac\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") "
Nov 03 21:48:18.289547 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.286244    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-maas-api-tls\") pod \"99af0d0b-bd91-4cc1-be41-d4414e335fac\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") "
Nov 03 21:48:18.289547 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.286278    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99af0d0b-bd91-4cc1-be41-d4414e335fac-kube-api-access-pn92t\" (UniqueName: \"kubernetes.io/projected/99af0d0b-bd91-4cc1-be41-d4414e335fac-kube-api-access-pn92t\") pod \"99af0d0b-bd91-4cc1-be41-d4414e335fac\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") "
Nov 03 21:48:18.289547 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.286371    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-metrics-certs\" (UniqueName: \"kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-metrics-certs\") pod \"99af0d0b-bd91-4cc1-be41-d4414e335fac\" (UID: \"99af0d0b-bd91-4cc1-be41-d4414e335fac\") "
Nov 03 21:48:18.294427 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.294375    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99af0d0b-bd91-4cc1-be41-d4414e335fac-openshift-service-ca" pod "99af0d0b-bd91-4cc1-be41-d4414e335fac" (UID: "99af0d0b-bd91-4cc1-be41-d4414e335fac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:18.316918 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.307009    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-metrics-certs" pod "99af0d0b-bd91-4cc1-be41-d4414e335fac" (UID: "99af0d0b-bd91-4cc1-be41-d4414e335fac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:18.316918 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.312680    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99af0d0b-bd91-4cc1-be41-d4414e335fac-kube-api-access-pn92t" pod "99af0d0b-bd91-4cc1-be41-d4414e335fac" (UID: "99af0d0b-bd91-4cc1-be41-d4414e335fac"). InnerVolumeSpecName "kube-api-access-pn92t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:18.400017 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.399351    4454 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-metrics-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:18.400017 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.399389    4454 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99af0d0b-bd91-4cc1-be41-d4414e335fac-openshift-service-ca\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:18.400017 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.399402    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn92t\" (UniqueName: \"kubernetes.io/projected/99af0d0b-bd91-4cc1-be41-d4414e335fac-kube-api-access-pn92t\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:18.405903 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.405855    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-maas-api-tls" pod "99af0d0b-bd91-4cc1-be41-d4414e335fac" (UID: "99af0d0b-bd91-4cc1-be41-d4414e335fac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:18.525148 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.518961    4454 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99af0d0b-bd91-4cc1-be41-d4414e335fac-maas-api-tls\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:18.566130 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.561796    4454 generic.go:356] "Generic (PLEG): container finished" podID="99af0d0b-bd91-4cc1-be41-d4414e335fac" containerID="62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9" exitCode=0
Nov 03 21:48:18.566130 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.561973    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv" event={"ID":"99af0d0b-bd91-4cc1-be41-d4414e335fac","Type":"ContainerDied","Data":"62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9"}
Nov 03 21:48:18.566130 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.562031    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv" event={"ID":"99af0d0b-bd91-4cc1-be41-d4414e335fac","Type":"ContainerDied","Data":"04f8843aaf1f958991fb94b2140e9c21b77a9bbaad8f8f5f6e8c28967f8378fe"}
Nov 03 21:48:18.566130 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.562062    4454 scope.go:122] "RemoveContainer" containerID="62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9"
Nov 03 21:48:18.566130 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.562337    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"
Nov 03 21:48:18.589477 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.589440    4454 scope.go:122] "RemoveContainer" containerID="62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9"
Nov 03 21:48:18.590632 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:18.590593    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9\": container with ID starting with 62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9 not found: ID does not exist" containerID="62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9"
Nov 03 21:48:18.590768 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.590638    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9"} err="failed to get container status \"62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9\": rpc error: code = NotFound desc = could not find container \"62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9\": container with ID starting with 62c9c3e4a3fc999039b68e0383f47c0f811ce2778f2da4eb20f9e535ce54fea9 not found: ID does not exist"
Nov 03 21:48:18.592575 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.592547    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:18.617084 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.617057    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"]
Nov 03 21:48:18.629414 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.629380    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-4eced5-5ff87bf9bd-q4ncv"]
Nov 03 21:48:18.719756 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.719712    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:18.730699 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.730668    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8" podUID="d6ab1abf-4dd9-43d2-845e-225243e8d9d6" containerName="payload-processing" containerID="cri-o://5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34" gracePeriod=25
Nov 03 21:48:18.754962 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:18.754928    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm" podUID="e376c103-f66d-4082-a27a-09bc2c48557a" containerName="payload-pre-processing" containerID="cri-o://a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32" gracePeriod=25
Nov 03 21:48:19.001015 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:19.000971    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:19.211174 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:19.211087    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8" podUID="d6ab1abf-4dd9-43d2-845e-225243e8d9d6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:48:19.230702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:19.230667    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99af0d0b-bd91-4cc1-be41-d4414e335fac" path="/var/lib/kubelet/pods/99af0d0b-bd91-4cc1-be41-d4414e335fac/volumes"
Nov 03 21:48:19.301071 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:19.301040    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:20.672631 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:20.672596    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"]
Nov 03 21:48:20.673132 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:20.672843    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb" podUID="12ae6e3f-92fb-43f4-9031-c91f75423aa4" containerName="maas-api" containerID="cri-o://2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0" gracePeriod=30
Nov 03 21:48:20.736241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:20.736205    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:21.076080 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.075569    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:48:21.173941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.173885    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:21.183197 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.183159    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-metrics-certs\") pod \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") "
Nov 03 21:48:21.183352 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.183312    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-maas-api-tls\") pod \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") "
Nov 03 21:48:21.183423 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.183359    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12ae6e3f-92fb-43f4-9031-c91f75423aa4-kube-api-access-knnhq\" (UniqueName: \"kubernetes.io/projected/12ae6e3f-92fb-43f4-9031-c91f75423aa4-kube-api-access-knnhq\") pod \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") "
Nov 03 21:48:21.183481 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.183470    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12ae6e3f-92fb-43f4-9031-c91f75423aa4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12ae6e3f-92fb-43f4-9031-c91f75423aa4-openshift-service-ca\") pod \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\" (UID: \"12ae6e3f-92fb-43f4-9031-c91f75423aa4\") "
Nov 03 21:48:21.185091 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.185026    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12ae6e3f-92fb-43f4-9031-c91f75423aa4-openshift-service-ca" pod "12ae6e3f-92fb-43f4-9031-c91f75423aa4" (UID: "12ae6e3f-92fb-43f4-9031-c91f75423aa4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:21.185910 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.185890    4454 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12ae6e3f-92fb-43f4-9031-c91f75423aa4-openshift-service-ca\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:21.188709 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.188670    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-maas-api-tls" pod "12ae6e3f-92fb-43f4-9031-c91f75423aa4" (UID: "12ae6e3f-92fb-43f4-9031-c91f75423aa4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:21.199626 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.199583    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-metrics-certs" pod "12ae6e3f-92fb-43f4-9031-c91f75423aa4" (UID: "12ae6e3f-92fb-43f4-9031-c91f75423aa4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:21.240122 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.239603    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12ae6e3f-92fb-43f4-9031-c91f75423aa4-kube-api-access-knnhq" pod "12ae6e3f-92fb-43f4-9031-c91f75423aa4" (UID: "12ae6e3f-92fb-43f4-9031-c91f75423aa4"). InnerVolumeSpecName "kube-api-access-knnhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:21.270376 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.270313    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" containerID="cri-o://0fa6aab340b84bca4bf2ca1354cbf5ecaaac22c67a2083d74d7d774c32ed5968" gracePeriod=45
Nov 03 21:48:21.275532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.275492    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:21.287030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.287004    4454 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-metrics-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:21.287030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.287030    4454 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12ae6e3f-92fb-43f4-9031-c91f75423aa4-maas-api-tls\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:21.287157 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.287043    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knnhq\" (UniqueName: \"kubernetes.io/projected/12ae6e3f-92fb-43f4-9031-c91f75423aa4-kube-api-access-knnhq\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:21.379943 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.379909    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:21.759882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.759839    4454 generic.go:356] "Generic (PLEG): container finished" podID="12ae6e3f-92fb-43f4-9031-c91f75423aa4" containerID="2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0" exitCode=0
Nov 03 21:48:21.760476 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.760453    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb" event={"ID":"12ae6e3f-92fb-43f4-9031-c91f75423aa4","Type":"ContainerDied","Data":"2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0"}
Nov 03 21:48:21.760660 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.760645    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb" event={"ID":"12ae6e3f-92fb-43f4-9031-c91f75423aa4","Type":"ContainerDied","Data":"5bb7288807444528649727afe4b82449367d8368e23338326539ade949c56dbd"}
Nov 03 21:48:21.760776 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.760764    4454 scope.go:122] "RemoveContainer" containerID="2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0"
Nov 03 21:48:21.761052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.761039    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"
Nov 03 21:48:21.777285 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.777254    4454 generic.go:356] "Generic (PLEG): container finished" podID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerID="0fa6aab340b84bca4bf2ca1354cbf5ecaaac22c67a2083d74d7d774c32ed5968" exitCode=0
Nov 03 21:48:21.787604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.787496    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" event={"ID":"7dcb03d9-851b-4209-934c-1d49f68da01b","Type":"ContainerDied","Data":"0fa6aab340b84bca4bf2ca1354cbf5ecaaac22c67a2083d74d7d774c32ed5968"}
Nov 03 21:48:21.861206 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.860679    4454 scope.go:122] "RemoveContainer" containerID="2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0"
Nov 03 21:48:21.861206 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:21.861132    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0\": container with ID starting with 2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0 not found: ID does not exist" containerID="2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0"
Nov 03 21:48:21.861206 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.861165    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0"} err="failed to get container status \"2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0\": rpc error: code = NotFound desc = could not find container \"2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0\": container with ID starting with 2ec2bbc9e7b2d94a15e1511393015d766424c234aceb8344bdeee32acfc28bf0 not found: ID does not exist"
Nov 03 21:48:21.866652 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.866629    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:48:21.905971 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.905941    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"]
Nov 03 21:48:21.908593 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.908557    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:21.913576 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:21.913489    4454 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb922f440_4072_4cfd_8cb1_f1a3646daaca.slice/crio-1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb922f440_4072_4cfd_8cb1_f1a3646daaca.slice/crio-conmon-1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538.scope\": RecentStats: unable to find data in memory cache]"
Nov 03 21:48:21.913974 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.913953    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-0a6da1-7cf4668747-mbztb"]
Nov 03 21:48:21.922967 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.922939    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-home\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-home\") pod \"7dcb03d9-851b-4209-934c-1d49f68da01b\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") "
Nov 03 21:48:21.923064 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.923051    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-dshm\") pod \"7dcb03d9-851b-4209-934c-1d49f68da01b\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") "
Nov 03 21:48:21.923124 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.923109    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-model-cache\") pod \"7dcb03d9-851b-4209-934c-1d49f68da01b\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") "
Nov 03 21:48:21.923210 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.923194    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7dcb03d9-851b-4209-934c-1d49f68da01b-kube-api-access-hzhxf\" (UniqueName: \"kubernetes.io/projected/7dcb03d9-851b-4209-934c-1d49f68da01b-kube-api-access-hzhxf\") pod \"7dcb03d9-851b-4209-934c-1d49f68da01b\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") "
Nov 03 21:48:21.923320 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.923301    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-home" pod "7dcb03d9-851b-4209-934c-1d49f68da01b" (UID: "7dcb03d9-851b-4209-934c-1d49f68da01b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:21.923388 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.923337    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-tmp-dir\") pod \"7dcb03d9-851b-4209-934c-1d49f68da01b\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") "
Nov 03 21:48:21.923435 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.923423    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7dcb03d9-851b-4209-934c-1d49f68da01b-tls-certs\" (UniqueName: \"kubernetes.io/secret/7dcb03d9-851b-4209-934c-1d49f68da01b-tls-certs\") pod \"7dcb03d9-851b-4209-934c-1d49f68da01b\" (UID: \"7dcb03d9-851b-4209-934c-1d49f68da01b\") "
Nov 03 21:48:21.923868 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.923844    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-model-cache" pod "7dcb03d9-851b-4209-934c-1d49f68da01b" (UID: "7dcb03d9-851b-4209-934c-1d49f68da01b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:21.924094 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.924066    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-tmp-dir" pod "7dcb03d9-851b-4209-934c-1d49f68da01b" (UID: "7dcb03d9-851b-4209-934c-1d49f68da01b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:21.924799 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.924771    4454 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-home\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:21.924799 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.924801    4454 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-model-cache\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:21.924933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.924816    4454 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-tmp-dir\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:21.927237 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.927211    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dcb03d9-851b-4209-934c-1d49f68da01b-kube-api-access-hzhxf" pod "7dcb03d9-851b-4209-934c-1d49f68da01b" (UID: "7dcb03d9-851b-4209-934c-1d49f68da01b"). InnerVolumeSpecName "kube-api-access-hzhxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:21.927906 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.927880    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-dshm" pod "7dcb03d9-851b-4209-934c-1d49f68da01b" (UID: "7dcb03d9-851b-4209-934c-1d49f68da01b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:21.929445 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:21.929420    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dcb03d9-851b-4209-934c-1d49f68da01b-tls-certs" pod "7dcb03d9-851b-4209-934c-1d49f68da01b" (UID: "7dcb03d9-851b-4209-934c-1d49f68da01b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:22.025950 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.025912    4454 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dcb03d9-851b-4209-934c-1d49f68da01b-tls-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:22.025950 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.025951    4454 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dcb03d9-851b-4209-934c-1d49f68da01b-dshm\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:22.026124 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.025965    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzhxf\" (UniqueName: \"kubernetes.io/projected/7dcb03d9-851b-4209-934c-1d49f68da01b-kube-api-access-hzhxf\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:22.027913 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.027878    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"
Nov 03 21:48:22.127176 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.127144    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b922f440-4072-4cfd-8cb1-f1a3646daaca-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b922f440-4072-4cfd-8cb1-f1a3646daaca-plugins-config-volume\") pod \"b922f440-4072-4cfd-8cb1-f1a3646daaca\" (UID: \"b922f440-4072-4cfd-8cb1-f1a3646daaca\") "
Nov 03 21:48:22.127347 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.127289    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b922f440-4072-4cfd-8cb1-f1a3646daaca-kube-api-access-99smx\" (UniqueName: \"kubernetes.io/projected/b922f440-4072-4cfd-8cb1-f1a3646daaca-kube-api-access-99smx\") pod \"b922f440-4072-4cfd-8cb1-f1a3646daaca\" (UID: \"b922f440-4072-4cfd-8cb1-f1a3646daaca\") "
Nov 03 21:48:22.127613 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.127588    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b922f440-4072-4cfd-8cb1-f1a3646daaca-plugins-config-volume" pod "b922f440-4072-4cfd-8cb1-f1a3646daaca" (UID: "b922f440-4072-4cfd-8cb1-f1a3646daaca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:22.127889 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.127871    4454 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b922f440-4072-4cfd-8cb1-f1a3646daaca-plugins-config-volume\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:22.136540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.133249    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b922f440-4072-4cfd-8cb1-f1a3646daaca-kube-api-access-99smx" pod "b922f440-4072-4cfd-8cb1-f1a3646daaca" (UID: "b922f440-4072-4cfd-8cb1-f1a3646daaca"). InnerVolumeSpecName "kube-api-access-99smx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:22.223525 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.223472    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:22.228887 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.228860    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99smx\" (UniqueName: \"kubernetes.io/projected/b922f440-4072-4cfd-8cb1-f1a3646daaca-kube-api-access-99smx\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:22.319380 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.319297    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:22.372881 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:22.372843    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:48:22.415832 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.415794    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:22.667241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.667210    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:22.779499 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.779464    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:22.796732 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.796699    4454 generic.go:356] "Generic (PLEG): container finished" podID="b922f440-4072-4cfd-8cb1-f1a3646daaca" containerID="1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538" exitCode=0
Nov 03 21:48:22.796888 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.796762    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"
Nov 03 21:48:22.796888 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.796776    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f" event={"ID":"b922f440-4072-4cfd-8cb1-f1a3646daaca","Type":"ContainerDied","Data":"1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538"}
Nov 03 21:48:22.796888 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.796824    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f" event={"ID":"b922f440-4072-4cfd-8cb1-f1a3646daaca","Type":"ContainerDied","Data":"29813e0fc3b69a1948e7367827edfe65d9330d56de423ca31dfaf7e4fde2ad09"}
Nov 03 21:48:22.796888 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.796858    4454 scope.go:122] "RemoveContainer" containerID="1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538"
Nov 03 21:48:22.799678 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.799656    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" event={"ID":"7dcb03d9-851b-4209-934c-1d49f68da01b","Type":"ContainerDied","Data":"1b5f34a8e3dcb7681c1ad29cd3fb1c61780ea31fbab63c9670cb1b78b37cc122"}
Nov 03 21:48:22.799678 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.799674    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"
Nov 03 21:48:22.807806 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.807781    4454 scope.go:122] "RemoveContainer" containerID="1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538"
Nov 03 21:48:22.808100 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:22.808075    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538\": container with ID starting with 1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538 not found: ID does not exist" containerID="1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538"
Nov 03 21:48:22.808173 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.808109    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538"} err="failed to get container status \"1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538\": rpc error: code = NotFound desc = could not find container \"1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538\": container with ID starting with 1b08f7bdc594c1010b15af646632f28cd93497522c1bca22be22769d9f039538 not found: ID does not exist"
Nov 03 21:48:22.808173 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.808126    4454 scope.go:122] "RemoveContainer" containerID="0fa6aab340b84bca4bf2ca1354cbf5ecaaac22c67a2083d74d7d774c32ed5968"
Nov 03 21:48:22.849291 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.849261    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:22.852930 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.852906    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"]
Nov 03 21:48:22.864177 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.864156    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-4eced5-5fddb9d9c5-znq7f"]
Nov 03 21:48:22.894476 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.894450    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"]
Nov 03 21:48:22.898762 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.898740    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-27fdf4/distinct-2-w2-27fdf4-kserve-659645cb79-qnczq"]
Nov 03 21:48:22.907995 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:22.907962    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:23.230970 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:23.230928    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12ae6e3f-92fb-43f4-9031-c91f75423aa4" path="/var/lib/kubelet/pods/12ae6e3f-92fb-43f4-9031-c91f75423aa4/volumes"
Nov 03 21:48:23.231492 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:23.231465    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" path="/var/lib/kubelet/pods/7dcb03d9-851b-4209-934c-1d49f68da01b/volumes"
Nov 03 21:48:23.232036 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:23.232015    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b922f440-4072-4cfd-8cb1-f1a3646daaca" path="/var/lib/kubelet/pods/b922f440-4072-4cfd-8cb1-f1a3646daaca/volumes"
Nov 03 21:48:24.263806 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:24.263770    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:25.744719 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.744688    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"]
Nov 03 21:48:25.745296 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745279    4454 cpu_manager.go:414] "containerMap: removing container" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745298    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745310    4454 cpu_manager.go:414] "containerMap: removing container" podUID="b922f440-4072-4cfd-8cb1-f1a3646daaca" containerName="payload-pre-processing"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745315    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b922f440-4072-4cfd-8cb1-f1a3646daaca" containerName="payload-pre-processing"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745325    4454 cpu_manager.go:414] "containerMap: removing container" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745330    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745341    4454 cpu_manager.go:414] "containerMap: removing container" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745346    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745352    4454 cpu_manager.go:414] "containerMap: removing container" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745356    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745364    4454 cpu_manager.go:414] "containerMap: removing container" podUID="33553b17-5b86-4d27-91d7-1d0d29a75280" containerName="payload-pre-processing"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745368    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33553b17-5b86-4d27-91d7-1d0d29a75280" containerName="payload-pre-processing"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745377    4454 cpu_manager.go:414] "containerMap: removing container" podUID="68b43a4e-3607-4620-b7a6-baf9c35cbf15" containerName="payload-pre-processing"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745382    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68b43a4e-3607-4620-b7a6-baf9c35cbf15" containerName="payload-pre-processing"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745391    4454 cpu_manager.go:414] "containerMap: removing container" podUID="28b54323-d6b0-40b7-ba3f-97ea85d99d18" containerName="istio-proxy"
Nov 03 21:48:25.745397 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745398    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28b54323-d6b0-40b7-ba3f-97ea85d99d18" containerName="istio-proxy"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745412    4454 cpu_manager.go:414] "containerMap: removing container" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745419    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745431    4454 cpu_manager.go:414] "containerMap: removing container" podUID="99af0d0b-bd91-4cc1-be41-d4414e335fac" containerName="maas-api"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745438    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99af0d0b-bd91-4cc1-be41-d4414e335fac" containerName="maas-api"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745447    4454 cpu_manager.go:414] "containerMap: removing container" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745451    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745460    4454 cpu_manager.go:414] "containerMap: removing container" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745466    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745480    4454 cpu_manager.go:414] "containerMap: removing container" podUID="c6c694f6-eeeb-4b68-89ca-64447fa9955d" containerName="istio-proxy"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745485    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6c694f6-eeeb-4b68-89ca-64447fa9955d" containerName="istio-proxy"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745492    4454 cpu_manager.go:414] "containerMap: removing container" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745496    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745521    4454 cpu_manager.go:414] "containerMap: removing container" podUID="12ae6e3f-92fb-43f4-9031-c91f75423aa4" containerName="maas-api"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745527    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12ae6e3f-92fb-43f4-9031-c91f75423aa4" containerName="maas-api"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745631    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="33553b17-5b86-4d27-91d7-1d0d29a75280" containerName="payload-pre-processing"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745642    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="68b43a4e-3607-4620-b7a6-baf9c35cbf15" containerName="payload-pre-processing"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745648    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="28b54323-d6b0-40b7-ba3f-97ea85d99d18" containerName="istio-proxy"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745655    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b4acf6f-77ba-468e-800e-0cf7e8a8a9b9" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745661    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="99af0d0b-bd91-4cc1-be41-d4414e335fac" containerName="maas-api"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745665    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="7dcb03d9-851b-4209-934c-1d49f68da01b" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745672    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="2007d8ff-83cc-42bc-9f9d-766b4ffda89b" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745676    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6c694f6-eeeb-4b68-89ca-64447fa9955d" containerName="istio-proxy"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745682    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2700b70-1f64-4db1-b6c2-552669059d93" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745687    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="12ae6e3f-92fb-43f4-9031-c91f75423aa4" containerName="maas-api"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745694    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="300b5ac5-db5d-4a3a-b00d-0c8dc0a637ac" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745701    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="b922f440-4072-4cfd-8cb1-f1a3646daaca" containerName="payload-pre-processing"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745706    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="6450e3ec-6e14-4520-94e9-3d41d5b578f8" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745711    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="f09c8d95-070e-4706-8ddd-92ddaea9c8d8" containerName="main"
Nov 03 21:48:25.745926 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.745715    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="063c1a56-0889-40ac-8eca-3f7acb93f7cb" containerName="main"
Nov 03 21:48:25.750572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.750549    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.763008 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.762983    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"]
Nov 03 21:48:25.766809 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.766787    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.774367 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.774346    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"]
Nov 03 21:48:25.805299 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.805263    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"]
Nov 03 21:48:25.849683 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.849648    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-certs\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.849834 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.849695    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-envoy\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.849834 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.849766    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4fb28f3-d432-4531-a424-b2063795e50f-istiod-ca-cert\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.849932 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.849894    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-data\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.849968 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.849930    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4fb28f3-d432-4531-a424-b2063795e50f-istio-podinfo\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.850011 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.849992    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-certs\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.850048 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.850034    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpm6b\" (UniqueName: \"kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-kube-api-access-hpm6b\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.850167 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.850151    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-data\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.850250 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.850174    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.850250 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.850234    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-token\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.850334 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.850297    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/772b7fa0-1b5d-4796-956f-2f5fca140669-istiod-ca-cert\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.850407 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.850386    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-istio-token\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.850463 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.850430    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.850525 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.850468    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-credential-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.850580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.850563    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-credential-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.850628 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.850592    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b78g\" (UniqueName: \"kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-kube-api-access-7b78g\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.850692 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.850678    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-envoy\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.850728 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.850713    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-podinfo\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.863860 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.863835    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"]
Nov 03 21:48:25.864163 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.864135    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm" podUID="35187015-8165-4126-8832-adcad32ca770" containerName="maas-api" containerID="cri-o://f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef" gracePeriod=30
Nov 03 21:48:25.872841 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.872772    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:25.952497 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.952462    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-podinfo\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.952685 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.952627    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-certs\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.952685 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.952658    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-envoy\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.952799 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.952689    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4fb28f3-d432-4531-a424-b2063795e50f-istiod-ca-cert\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.952799 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.952755    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-data\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.952799 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.952787    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4fb28f3-d432-4531-a424-b2063795e50f-istio-podinfo\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.952944 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.952825    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-certs\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.952944 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.952867    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpm6b\" (UniqueName: \"kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-kube-api-access-hpm6b\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.952944 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.952932    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-data\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.953080 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.952959    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.953080 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953030    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-token\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.953080 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953040    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-certs\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.953080 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953070    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/772b7fa0-1b5d-4796-956f-2f5fca140669-istiod-ca-cert\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.953254 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953095    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-data\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.953254 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953100    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-istio-token\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.953254 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953157    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.953254 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953212    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-credential-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.953436 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953271    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-credential-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.953436 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953281    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-certs\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.953436 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953296    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-data\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.953436 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953299    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b78g\" (UniqueName: \"kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-kube-api-access-7b78g\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.953660 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953566    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.953826 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953803    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.954144 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953966    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-envoy\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.954144 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.953890    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-credential-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.954272 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.954164    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/772b7fa0-1b5d-4796-956f-2f5fca140669-istiod-ca-cert\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.954272 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.954207    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-credential-socket\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.955011 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.954982    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4fb28f3-d432-4531-a424-b2063795e50f-istiod-ca-cert\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.955351 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.955326    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-envoy\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.956235 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.956214    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-envoy\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.956307 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.956240    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4fb28f3-d432-4531-a424-b2063795e50f-istio-podinfo\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.956699 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.956684    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-podinfo\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.972132 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.972101    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-token\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:25.972406 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.972380    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:25.974083 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.973630    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpm6b\" (UniqueName: \"kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-kube-api-access-hpm6b\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.977102 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.976887    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-istio-token\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:25.977102 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:25.977098    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b78g\" (UniqueName: \"kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-kube-api-access-7b78g\") pod \"e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") " pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:26.066238 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.066203    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:26.082112 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.081907    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:26.121969 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.121935    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:26.133643 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.133586    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:48:26.227552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.227497    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:26.258726 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.258698    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-metrics-certs\" (UniqueName: \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-metrics-certs\") pod \"35187015-8165-4126-8832-adcad32ca770\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") "
Nov 03 21:48:26.258882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.258762    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35187015-8165-4126-8832-adcad32ca770-kube-api-access-zllpj\" (UniqueName: \"kubernetes.io/projected/35187015-8165-4126-8832-adcad32ca770-kube-api-access-zllpj\") pod \"35187015-8165-4126-8832-adcad32ca770\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") "
Nov 03 21:48:26.258882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.258825    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35187015-8165-4126-8832-adcad32ca770-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35187015-8165-4126-8832-adcad32ca770-openshift-service-ca\") pod \"35187015-8165-4126-8832-adcad32ca770\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") "
Nov 03 21:48:26.258950 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.258889    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-maas-api-tls\") pod \"35187015-8165-4126-8832-adcad32ca770\" (UID: \"35187015-8165-4126-8832-adcad32ca770\") "
Nov 03 21:48:26.259275 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.259243    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35187015-8165-4126-8832-adcad32ca770-openshift-service-ca" pod "35187015-8165-4126-8832-adcad32ca770" (UID: "35187015-8165-4126-8832-adcad32ca770"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:26.259589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.259569    4454 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35187015-8165-4126-8832-adcad32ca770-openshift-service-ca\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:26.260911 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.260885    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-metrics-certs" pod "35187015-8165-4126-8832-adcad32ca770" (UID: "35187015-8165-4126-8832-adcad32ca770"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:26.261004 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.260944    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-maas-api-tls" pod "35187015-8165-4126-8832-adcad32ca770" (UID: "35187015-8165-4126-8832-adcad32ca770"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:26.261004 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.260973    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35187015-8165-4126-8832-adcad32ca770-kube-api-access-zllpj" pod "35187015-8165-4126-8832-adcad32ca770" (UID: "35187015-8165-4126-8832-adcad32ca770"). InnerVolumeSpecName "kube-api-access-zllpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:26.369763 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.366827    4454 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-metrics-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:26.369763 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.366867    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zllpj\" (UniqueName: \"kubernetes.io/projected/35187015-8165-4126-8832-adcad32ca770-kube-api-access-zllpj\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:26.369763 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.366883    4454 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35187015-8165-4126-8832-adcad32ca770-maas-api-tls\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:26.444572 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.444408    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"]
Nov 03 21:48:26.490316 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.486721    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"]
Nov 03 21:48:26.490316 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.487018    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz" podUID="3d31fd0d-3343-45cb-aa88-297d86d9ddfb" containerName="istio-proxy" containerID="cri-o://e959e612e10ffeec2370d5861c1221a35af42015f4dd802e6a9fcd706619e5a4" gracePeriod=30
Nov 03 21:48:26.585561 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.576180    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"]
Nov 03 21:48:26.585561 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.576490    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" podUID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" containerName="istio-proxy" containerID="cri-o://e2698f86babe330d859ed777b1492d077061f4ef543d0da686d65b17d8950774" gracePeriod=30
Nov 03 21:48:26.585561 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.583116    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:26.600371 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.600340    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"]
Nov 03 21:48:26.600744 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.600723    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"]
Nov 03 21:48:26.604944 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.604907    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:48:26.605057 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.604987    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:48:26.605057 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.605028    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:48:26.608254 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.606702    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:48:26.608254 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.606766    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:48:26.608254 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.606806    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 21:48:26.753815 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.750596    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:26.848427 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.848341    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:26.960238 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.960205    4454 generic.go:356] "Generic (PLEG): container finished" podID="35187015-8165-4126-8832-adcad32ca770" containerID="f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef" exitCode=0
Nov 03 21:48:26.960430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.960311    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm" event={"ID":"35187015-8165-4126-8832-adcad32ca770","Type":"ContainerDied","Data":"f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef"}
Nov 03 21:48:26.960430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.960363    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm" event={"ID":"35187015-8165-4126-8832-adcad32ca770","Type":"ContainerDied","Data":"da228def14aa69f097e2f905514f264cd6257f23738e1e1cc76870099aa44533"}
Nov 03 21:48:26.960430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.960391    4454 scope.go:122] "RemoveContainer" containerID="f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef"
Nov 03 21:48:26.961461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.960288    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"
Nov 03 21:48:26.961872 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.961816    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:26.964183 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.964149    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" event={"ID":"f4fb28f3-d432-4531-a424-b2063795e50f","Type":"ContainerStarted","Data":"ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9"}
Nov 03 21:48:26.964294 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.964194    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" event={"ID":"f4fb28f3-d432-4531-a424-b2063795e50f","Type":"ContainerStarted","Data":"065dc3130c9f588489146cdaad4d9f912fd519129d019aa0b987dadb5d32d719"}
Nov 03 21:48:26.967350 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.967324    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy" containerID="cri-o://dd7b09cf846413837cac64c2a533158cad3571614de2ffea80120e719b758bb4" gracePeriod=30
Nov 03 21:48:26.968371 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.968348    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" event={"ID":"772b7fa0-1b5d-4796-956f-2f5fca140669","Type":"ContainerStarted","Data":"fdff37e7c6734683664812b09f993663fb8df038db4f7d29db3107e2fc167f6e"}
Nov 03 21:48:26.968546 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.968526    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" event={"ID":"772b7fa0-1b5d-4796-956f-2f5fca140669","Type":"ContainerStarted","Data":"e0f0160ef6886886ff05a76da5691262d958ed4a0bfe952431745e73302037e0"}
Nov 03 21:48:26.976661 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.976644    4454 scope.go:122] "RemoveContainer" containerID="f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef"
Nov 03 21:48:26.977073 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:26.977051    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef\": container with ID starting with f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef not found: ID does not exist" containerID="f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef"
Nov 03 21:48:26.977208 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:26.977185    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef"} err="failed to get container status \"f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef\": rpc error: code = NotFound desc = could not find container \"f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef\": container with ID starting with f116ed3d8b4e89bcaa692b1ccc48709d364b4942d9c0c159d0bc810f0c18b2ef not found: ID does not exist"
Nov 03 21:48:27.028135 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.027800    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" podStartSLOduration=2.027781923 podStartE2EDuration="2.027781923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:48:27.025220452 +0000 UTC m=+1115.904042754" watchObservedRunningTime="2026-11-03 21:48:27.027781923 +0000 UTC m=+1115.906604208"
Nov 03 21:48:27.067982 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.067929    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:27.068294 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.068253    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" podStartSLOduration=2.068240019 podStartE2EDuration="2.068240019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:48:27.064934547 +0000 UTC m=+1115.943756853" watchObservedRunningTime="2026-11-03 21:48:27.068240019 +0000 UTC m=+1115.947062302"
Nov 03 21:48:27.069375 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.069348    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 21:48:27.069459 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.069402    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 21:48:27.082578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.082553    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:27.083869 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.083847    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 03 21:48:27.083972 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.083890    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 03 21:48:27.181774 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.181569    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"]
Nov 03 21:48:27.206041 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.205992    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-05bc27-5d4f5cf998-gvwwm"]
Nov 03 21:48:27.235576 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.235544    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35187015-8165-4126-8832-adcad32ca770" path="/var/lib/kubelet/pods/35187015-8165-4126-8832-adcad32ca770/volumes"
Nov 03 21:48:27.392683 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.392649    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:27.721634 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.721574    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:27.839576 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:27.828967    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:28.069026 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:28.068941    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 21:48:28.069280 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:28.069246    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 21:48:28.085925 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:28.085881    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 03 21:48:28.086187 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:28.086157    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 03 21:48:28.221674 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:28.221613    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm" podUID="e376c103-f66d-4082-a27a-09bc2c48557a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:48:28.251383 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:28.248274    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:48:28.730916 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:28.730884    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:29.057890 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:29.047956    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:29.149532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:29.077368    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 21:48:29.209531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:29.192543    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:29.209531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:29.150135    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 03 21:48:29.209531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:29.208818    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 03 21:48:29.249530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:29.228925    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 21:48:29.279535 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:29.279468    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8" podUID="d6ab1abf-4dd9-43d2-845e-225243e8d9d6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:48:29.568871 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:29.568833    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:30.075578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:30.074371    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 21:48:30.075578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:30.074434    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 21:48:30.107547 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:30.093307    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 03 21:48:30.107547 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:30.093362    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 03 21:48:30.272531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:30.270146    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:30.391540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:30.391110    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:30.559531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:30.558825    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"]
Nov 03 21:48:30.559531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:30.559075    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerName="istio-proxy" containerID="cri-o://f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3" gracePeriod=30
Nov 03 21:48:31.030534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.006045    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:31.186523 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.154174    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:48:31.194039 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.187049    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:31.241533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.239831    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 03 21:48:31.241533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.240677    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 03 21:48:31.361877 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.361825    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 21:48:31.362229 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.362200    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 21:48:31.435532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.435083    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:31.751790 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.751717    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"]
Nov 03 21:48:31.752041 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.751879    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"]
Nov 03 21:48:31.765957 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.759605    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"]
Nov 03 21:48:31.765957 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.759665    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"]
Nov 03 21:48:31.765957 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.759894    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy" containerID="cri-o://ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d" gracePeriod=30
Nov 03 21:48:31.765957 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.760173    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerName="istio-proxy" containerID="cri-o://2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545" gracePeriod=30
Nov 03 21:48:31.765957 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.760359    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy" containerID="cri-o://d774c87dcd02ee26c5b237661769506c1351fda91c42e3863169616aab1bde70" gracePeriod=30
Nov 03 21:48:31.766888 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:31.766229    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy" containerID="cri-o://b22f14209489eb3100c547e4ee76a85724471e18170cf26a73ccf16504afb7d0" gracePeriod=30
Nov 03 21:48:32.053805 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.023642    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:32.127469 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.109804    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 21:48:32.127469 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.109868    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 21:48:32.275882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.275778    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:48:32.280432 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.279301    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:48:32.385589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.382624    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 03 21:48:32.385589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.382684    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 03 21:48:32.470976 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.470933    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:48:32.471289 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.471254    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:32.756525 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.756476    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:48:32.756799 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.756753    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz" podUID="3d31fd0d-3343-45cb-aa88-297d86d9ddfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:48:32.838550 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.838498    4454 generic.go:356] "Generic (PLEG): container finished" podID="3d31fd0d-3343-45cb-aa88-297d86d9ddfb" containerID="e959e612e10ffeec2370d5861c1221a35af42015f4dd802e6a9fcd706619e5a4" exitCode=0
Nov 03 21:48:32.838912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.838790    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz" event={"ID":"3d31fd0d-3343-45cb-aa88-297d86d9ddfb","Type":"ContainerDied","Data":"e959e612e10ffeec2370d5861c1221a35af42015f4dd802e6a9fcd706619e5a4"}
Nov 03 21:48:32.852193 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.852161    4454 generic.go:356] "Generic (PLEG): container finished" podID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" containerID="e2698f86babe330d859ed777b1492d077061f4ef543d0da686d65b17d8950774" exitCode=0
Nov 03 21:48:32.852531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.852489    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" event={"ID":"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad","Type":"ContainerDied","Data":"e2698f86babe330d859ed777b1492d077061f4ef543d0da686d65b17d8950774"}
Nov 03 21:48:32.887764 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.887715    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:48:32.888019 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:32.887993    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" podUID="5c1f09c7-5b84-4789-9b2b-17657863cd18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:33.240217 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.222986    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 03 21:48:33.240217 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.223044    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 03 21:48:33.240217 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.223452    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 21:48:33.240217 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.223490    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 21:48:33.241406 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.241189    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:33.261247 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.260460    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn" podUID="f9a60514-d1ae-4072-b77a-9eb51ff05a08" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:33.281557 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.279667    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:48:33.281557 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.281372    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:33.591607 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.587199    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:48:33.591607 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.587265    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:48:33.632011 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.602572    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:48:33.691547 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.691127    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:48:33.691547 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.691190    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" podUID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:33.695552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.692460    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istiod-ca-cert\") pod \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") "
Nov 03 21:48:33.695552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.692568    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-credential-socket\") pod \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") "
Nov 03 21:48:33.695552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.692644    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-certs\") pod \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") "
Nov 03 21:48:33.695552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.692720    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-token\" (UniqueName: \"kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-token\") pod \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") "
Nov 03 21:48:33.695552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.692755    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-data\") pod \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") "
Nov 03 21:48:33.695552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.692819    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-kube-api-access-9p7vw\" (UniqueName: \"kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-kube-api-access-9p7vw\") pod \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") "
Nov 03 21:48:33.695552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.692851    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-socket\") pod \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") "
Nov 03 21:48:33.695552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.692990    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-podinfo\") pod \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") "
Nov 03 21:48:33.695552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.693027    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-envoy\") pod \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\" (UID: \"3d31fd0d-3343-45cb-aa88-297d86d9ddfb\") "
Nov 03 21:48:33.727294 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.727078    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-credential-socket" pod "3d31fd0d-3343-45cb-aa88-297d86d9ddfb" (UID: "3d31fd0d-3343-45cb-aa88-297d86d9ddfb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:33.727850 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.727613    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istiod-ca-cert" pod "3d31fd0d-3343-45cb-aa88-297d86d9ddfb" (UID: "3d31fd0d-3343-45cb-aa88-297d86d9ddfb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:33.728350 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.728050    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-certs" pod "3d31fd0d-3343-45cb-aa88-297d86d9ddfb" (UID: "3d31fd0d-3343-45cb-aa88-297d86d9ddfb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:33.729304 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.729131    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-data" pod "3d31fd0d-3343-45cb-aa88-297d86d9ddfb" (UID: "3d31fd0d-3343-45cb-aa88-297d86d9ddfb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:33.729750 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.729457    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-socket" pod "3d31fd0d-3343-45cb-aa88-297d86d9ddfb" (UID: "3d31fd0d-3343-45cb-aa88-297d86d9ddfb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:33.742109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.731655    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-token" pod "3d31fd0d-3343-45cb-aa88-297d86d9ddfb" (UID: "3d31fd0d-3343-45cb-aa88-297d86d9ddfb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:33.749611 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.747628    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-podinfo" pod "3d31fd0d-3343-45cb-aa88-297d86d9ddfb" (UID: "3d31fd0d-3343-45cb-aa88-297d86d9ddfb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:33.749611 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.747753    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-kube-api-access-9p7vw" pod "3d31fd0d-3343-45cb-aa88-297d86d9ddfb" (UID: "3d31fd0d-3343-45cb-aa88-297d86d9ddfb"). InnerVolumeSpecName "kube-api-access-9p7vw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:33.798545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.795748    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p7vw\" (UniqueName: \"kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-kube-api-access-9p7vw\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:33.798545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.795786    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:33.798545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.795805    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:33.798545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.795818    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:33.798545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.795833    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:33.798545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.795846    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:33.798545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.795860    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:33.798545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.795874    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:33.904539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.864721    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-envoy" pod "3d31fd0d-3343-45cb-aa88-297d86d9ddfb" (UID: "3d31fd0d-3343-45cb-aa88-297d86d9ddfb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:33.904539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.881917    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:33.904539 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.897459    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d31fd0d-3343-45cb-aa88-297d86d9ddfb-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:33.968838 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.968802    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz" event={"ID":"3d31fd0d-3343-45cb-aa88-297d86d9ddfb","Type":"ContainerDied","Data":"4fcb75d5e9535288e70357d72be16732076b0037952d56a042c0611a88274a0d"}
Nov 03 21:48:33.969103 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.969085    4454 scope.go:122] "RemoveContainer" containerID="e959e612e10ffeec2370d5861c1221a35af42015f4dd802e6a9fcd706619e5a4"
Nov 03 21:48:33.969399 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:33.969381    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"
Nov 03 21:48:34.075978 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.075936    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:34.077297 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.077269    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 21:48:34.077527 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.077488    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 21:48:34.113530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.091412    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 03 21:48:34.113530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.091487    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 03 21:48:34.161317 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.161222    4454 generic.go:356] "Generic (PLEG): container finished" podID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerID="dd7b09cf846413837cac64c2a533158cad3571614de2ffea80120e719b758bb4" exitCode=0
Nov 03 21:48:34.161898 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.161870    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" event={"ID":"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515","Type":"ContainerDied","Data":"dd7b09cf846413837cac64c2a533158cad3571614de2ffea80120e719b758bb4"}
Nov 03 21:48:34.176795 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.170580    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.247110    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn"
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.252209    4454 scope.go:122] "RemoveContainer" containerID="ddecd189b98992a329b97f08595bf60f2970d964a525fb120b82ed9ce146b5e2"
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.252734    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-envoy\") pod \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") "
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.252829    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-podinfo\") pod \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") "
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.252867    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-credential-socket\") pod \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") "
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.252976    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-token\" (UniqueName: \"kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-token\") pod \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") "
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.253036    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-kube-api-access-hmtht\" (UniqueName: \"kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-kube-api-access-hmtht\") pod \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") "
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.253138    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istiod-ca-cert\") pod \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") "
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.253167    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-socket\") pod \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") "
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.253196    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-data\") pod \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") "
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.253221    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-certs\") pod \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\" (UID: \"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad\") "
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.254002    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-certs" pod "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" (UID: "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.257231    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-socket" pod "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" (UID: "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.257389    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.257526    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istiod-ca-cert" pod "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" (UID: "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.257643    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-credential-socket" pod "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" (UID: "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.258351    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-data" pod "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" (UID: "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:34.259543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.259456    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-envoy" pod "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" (UID: "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:34.345318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.345277    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-podinfo" pod "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" (UID: "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:34.394536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.367854    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"]
Nov 03 21:48:34.394536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.367915    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"]
Nov 03 21:48:34.394536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.368094    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerName="istio-proxy" containerID="cri-o://bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b" gracePeriod=30
Nov 03 21:48:34.394536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.368290    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" podUID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" containerName="istio-proxy" containerID="cri-o://f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2" gracePeriod=30
Nov 03 21:48:34.394536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.368701    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:34.394536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.368722    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:34.394536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.368738    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:34.394536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.368750    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:34.394536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.368763    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:34.394536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.368777    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:34.394536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.368789    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:34.415970 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.415865    4454 patch_prober.go:29] interesting pod/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:48:34.572541 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.512822    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" podUID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:48:34.572541 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.513081    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-token" pod "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" (UID: "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:34.572541 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.419171    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"]
Nov 03 21:48:34.572541 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.513497    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"]
Nov 03 21:48:34.572541 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.513792    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" containerName="istio-proxy" containerID="cri-o://ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9" gracePeriod=30
Nov 03 21:48:34.572541 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.464667    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-kube-api-access-hmtht" pod "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" (UID: "5eda1eb4-b61c-44fd-ad6c-d11d122f36ad"). InnerVolumeSpecName "kube-api-access-hmtht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:34.572541 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.514207    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerName="istio-proxy" containerID="cri-o://fdff37e7c6734683664812b09f993663fb8df038db4f7d29db3107e2fc167f6e" gracePeriod=30
Nov 03 21:48:34.572541 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.537170    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:34.572541 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.537203    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmtht\" (UniqueName: \"kubernetes.io/projected/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad-kube-api-access-hmtht\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:34.588623 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.575648    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:34.736557 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.707038    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:48:34.756704 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.756672    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"]
Nov 03 21:48:34.763530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.760583    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-zkzrz"]
Nov 03 21:48:34.796680 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.796644    4454 patch_prober.go:29] interesting pod/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:48:34.796936 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.796911    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:48:34.887532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.886333    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-podinfo\") pod \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") "
Nov 03 21:48:34.887532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.886463    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-kube-api-access-8brkm\" (UniqueName: \"kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-kube-api-access-8brkm\") pod \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") "
Nov 03 21:48:34.887532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.886932    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-credential-socket\") pod \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") "
Nov 03 21:48:34.887532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.887048    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-socket\") pod \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") "
Nov 03 21:48:34.887532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.887090    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-token\" (UniqueName: \"kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-token\") pod \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") "
Nov 03 21:48:34.887532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.887147    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istiod-ca-cert\") pod \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") "
Nov 03 21:48:34.887532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.887179    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-data\") pod \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") "
Nov 03 21:48:34.887532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.887209    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-envoy\") pod \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") "
Nov 03 21:48:34.887532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.887238    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-certs\") pod \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\" (UID: \"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515\") "
Nov 03 21:48:34.889273 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.888990    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istiod-ca-cert" pod "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" (UID: "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:34.889530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.889444    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-data" pod "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" (UID: "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:34.891163 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.891007    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-credential-socket" pod "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" (UID: "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:34.900383 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.891305    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-socket" pod "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" (UID: "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:34.902178 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.902148    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-podinfo" pod "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" (UID: "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:34.908189 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.908159    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-certs" pod "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" (UID: "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:34.918625 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.918594    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-envoy" pod "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" (UID: "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:34.919602 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.919575    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-kube-api-access-8brkm" pod "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" (UID: "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515"). InnerVolumeSpecName "kube-api-access-8brkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:34.931915 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:34.930739    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-token" pod "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" (UID: "dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:35.004367 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.004324    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:35.019529 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.018167    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8brkm\" (UniqueName: \"kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-kube-api-access-8brkm\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:35.019788 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.019766    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:35.019922 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.019906    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:35.020028 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.020016    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:35.020149 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.020135    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:35.020267 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.020255    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:35.020373 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.020362    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:35.020470 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.020460    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:35.174990 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.174936    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2" event={"ID":"dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515","Type":"ContainerDied","Data":"e80493a7ee7dd09a04a53d799ea613bc5796a0d0e12f3e145ac4bf40288cc12e"}
Nov 03 21:48:35.175297 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.175260    4454 scope.go:122] "RemoveContainer" containerID="dd7b09cf846413837cac64c2a533158cad3571614de2ffea80120e719b758bb4"
Nov 03 21:48:35.175750 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.175733    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"
Nov 03 21:48:35.193312 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.193278    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964" event={"ID":"5eda1eb4-b61c-44fd-ad6c-d11d122f36ad","Type":"ContainerDied","Data":"be821d1f0f7c4ba66cebbf5dc6fe4d66d5bebbb3f7a2f0b8b2c828ca9d076bae"}
Nov 03 21:48:35.193625 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.193607    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"
Nov 03 21:48:35.240290 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.240261    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d31fd0d-3343-45cb-aa88-297d86d9ddfb" path="/var/lib/kubelet/pods/3d31fd0d-3343-45cb-aa88-297d86d9ddfb/volumes"
Nov 03 21:48:35.247317 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.247285    4454 patch_prober.go:29] interesting pod/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:48:35.247479 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.247338    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:48:35.247892 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.247711    4454 scope.go:122] "RemoveContainer" containerID="e2698f86babe330d859ed777b1492d077061f4ef543d0da686d65b17d8950774"
Nov 03 21:48:35.338773 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.337229    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"]
Nov 03 21:48:35.339634 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.339610    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-pc964"]
Nov 03 21:48:35.387601 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.384057    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"]
Nov 03 21:48:35.388739 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:35.388712    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-4eced5-openshift-default-75c5d85f9f-bslh2"]
Nov 03 21:48:36.218103 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:36.218068    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:36.218668 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:36.218645    4454 patch_prober.go:29] interesting pod/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:48:36.218838 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:36.218815    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:48:36.843667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:36.843627    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:36.942597 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:36.942407    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:37.049707 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.049679    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:48:37.190551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.190083    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-podinfo\") pod \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") "
Nov 03 21:48:37.190551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.190186    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-envoy\") pod \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") "
Nov 03 21:48:37.190551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.190222    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-kube-api-access-pbxf6\" (UniqueName: \"kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-kube-api-access-pbxf6\") pod \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") "
Nov 03 21:48:37.190551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.190272    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-token\" (UniqueName: \"kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-token\") pod \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") "
Nov 03 21:48:37.190551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.190357    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-data\") pod \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") "
Nov 03 21:48:37.190551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.190404    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-credential-socket\") pod \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") "
Nov 03 21:48:37.190551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.190480    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-socket\") pod \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") "
Nov 03 21:48:37.191532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.191235    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-data" pod "ad15d2cf-954f-4711-a95f-5ce2a24fd12f" (UID: "ad15d2cf-954f-4711-a95f-5ce2a24fd12f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.191532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.191255    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istiod-ca-cert\") pod \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") "
Nov 03 21:48:37.191532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.191350    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-certs\") pod \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\" (UID: \"ad15d2cf-954f-4711-a95f-5ce2a24fd12f\") "
Nov 03 21:48:37.191940 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.191915    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.192193 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.192165    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-certs" pod "ad15d2cf-954f-4711-a95f-5ce2a24fd12f" (UID: "ad15d2cf-954f-4711-a95f-5ce2a24fd12f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.192408 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.192385    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-credential-socket" pod "ad15d2cf-954f-4711-a95f-5ce2a24fd12f" (UID: "ad15d2cf-954f-4711-a95f-5ce2a24fd12f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.192632 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.192610    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-socket" pod "ad15d2cf-954f-4711-a95f-5ce2a24fd12f" (UID: "ad15d2cf-954f-4711-a95f-5ce2a24fd12f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.192969 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.192938    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istiod-ca-cert" pod "ad15d2cf-954f-4711-a95f-5ce2a24fd12f" (UID: "ad15d2cf-954f-4711-a95f-5ce2a24fd12f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:37.208931 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.197859    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-token" pod "ad15d2cf-954f-4711-a95f-5ce2a24fd12f" (UID: "ad15d2cf-954f-4711-a95f-5ce2a24fd12f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:37.208931 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.197976    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-kube-api-access-pbxf6" pod "ad15d2cf-954f-4711-a95f-5ce2a24fd12f" (UID: "ad15d2cf-954f-4711-a95f-5ce2a24fd12f"). InnerVolumeSpecName "kube-api-access-pbxf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:37.208931 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.208894    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-podinfo" pod "ad15d2cf-954f-4711-a95f-5ce2a24fd12f" (UID: "ad15d2cf-954f-4711-a95f-5ce2a24fd12f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:37.210814 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.210779    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-envoy" pod "ad15d2cf-954f-4711-a95f-5ce2a24fd12f" (UID: "ad15d2cf-954f-4711-a95f-5ce2a24fd12f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.226230 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.225772    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:48:37.249048 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.248584    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" path="/var/lib/kubelet/pods/5eda1eb4-b61c-44fd-ad6c-d11d122f36ad/volumes"
Nov 03 21:48:37.249536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.249314    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" path="/var/lib/kubelet/pods/dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515/volumes"
Nov 03 21:48:37.251001 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.250976    4454 generic.go:356] "Generic (PLEG): container finished" podID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerID="ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d" exitCode=0
Nov 03 21:48:37.251441 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.251204    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"
Nov 03 21:48:37.265406 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.253062    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" event={"ID":"ad15d2cf-954f-4711-a95f-5ce2a24fd12f","Type":"ContainerDied","Data":"ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d"}
Nov 03 21:48:37.267596 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.267567    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4" event={"ID":"ad15d2cf-954f-4711-a95f-5ce2a24fd12f","Type":"ContainerDied","Data":"d9a16cf8533d06298f315ff9a2ca2ecb12d3ed29a43a018f8249356e8d0b715e"}
Nov 03 21:48:37.267758 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.267738    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" event={"ID":"2588bcca-4842-455d-a4fd-7397fa0bdb5f","Type":"ContainerDied","Data":"d774c87dcd02ee26c5b237661769506c1351fda91c42e3863169616aab1bde70"}
Nov 03 21:48:37.267900 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.267885    4454 scope.go:122] "RemoveContainer" containerID="ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d"
Nov 03 21:48:37.268939 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.259459    4454 generic.go:356] "Generic (PLEG): container finished" podID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerID="d774c87dcd02ee26c5b237661769506c1351fda91c42e3863169616aab1bde70" exitCode=0
Nov 03 21:48:37.285801 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.277263    4454 generic.go:356] "Generic (PLEG): container finished" podID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerID="2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545" exitCode=0
Nov 03 21:48:37.285801 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.277389    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" event={"ID":"21f7c684-2bb5-4c67-9009-1d9f854be081","Type":"ContainerDied","Data":"2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545"}
Nov 03 21:48:37.285801 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.277434    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n" event={"ID":"21f7c684-2bb5-4c67-9009-1d9f854be081","Type":"ContainerDied","Data":"073d62a54079d2bbe92adb9bffa999bbf4f52b9d408c32c342da4aac5200d26e"}
Nov 03 21:48:37.285801 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.277535    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"
Nov 03 21:48:37.293960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.292812    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-token\" (UniqueName: \"kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-token\") pod \"21f7c684-2bb5-4c67-9009-1d9f854be081\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") "
Nov 03 21:48:37.293960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.292919    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-data\") pod \"21f7c684-2bb5-4c67-9009-1d9f854be081\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") "
Nov 03 21:48:37.293960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.293000    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-podinfo\") pod \"21f7c684-2bb5-4c67-9009-1d9f854be081\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") "
Nov 03 21:48:37.293960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.293073    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-socket\") pod \"21f7c684-2bb5-4c67-9009-1d9f854be081\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") "
Nov 03 21:48:37.293960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.293121    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-credential-socket\") pod \"21f7c684-2bb5-4c67-9009-1d9f854be081\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") "
Nov 03 21:48:37.293960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.293176    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-envoy\") pod \"21f7c684-2bb5-4c67-9009-1d9f854be081\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") "
Nov 03 21:48:37.293960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.293211    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-certs\") pod \"21f7c684-2bb5-4c67-9009-1d9f854be081\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") "
Nov 03 21:48:37.293960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.293287    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21f7c684-2bb5-4c67-9009-1d9f854be081-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21f7c684-2bb5-4c67-9009-1d9f854be081-istiod-ca-cert\") pod \"21f7c684-2bb5-4c67-9009-1d9f854be081\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") "
Nov 03 21:48:37.293960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.293331    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-kube-api-access-gttzk\" (UniqueName: \"kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-kube-api-access-gttzk\") pod \"21f7c684-2bb5-4c67-9009-1d9f854be081\" (UID: \"21f7c684-2bb5-4c67-9009-1d9f854be081\") "
Nov 03 21:48:37.293960 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.293654    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-socket" pod "21f7c684-2bb5-4c67-9009-1d9f854be081" (UID: "21f7c684-2bb5-4c67-9009-1d9f854be081"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.294658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.294308    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.294658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.294330    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.294658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.294346    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.294658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.294404    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.294658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.294422    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.294658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.294435    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.294658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.294448    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.294658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.294464    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.294658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.294479    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbxf6\" (UniqueName: \"kubernetes.io/projected/ad15d2cf-954f-4711-a95f-5ce2a24fd12f-kube-api-access-pbxf6\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.297219 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.295677    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-data" pod "21f7c684-2bb5-4c67-9009-1d9f854be081" (UID: "21f7c684-2bb5-4c67-9009-1d9f854be081"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.297219 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.296771    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-certs" pod "21f7c684-2bb5-4c67-9009-1d9f854be081" (UID: "21f7c684-2bb5-4c67-9009-1d9f854be081"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.297219 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.297009    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-envoy" pod "21f7c684-2bb5-4c67-9009-1d9f854be081" (UID: "21f7c684-2bb5-4c67-9009-1d9f854be081"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.297219 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.297136    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21f7c684-2bb5-4c67-9009-1d9f854be081-istiod-ca-cert" pod "21f7c684-2bb5-4c67-9009-1d9f854be081" (UID: "21f7c684-2bb5-4c67-9009-1d9f854be081"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:37.298195 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.298076    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-credential-socket" pod "21f7c684-2bb5-4c67-9009-1d9f854be081" (UID: "21f7c684-2bb5-4c67-9009-1d9f854be081"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.300317 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.300280    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-podinfo" pod "21f7c684-2bb5-4c67-9009-1d9f854be081" (UID: "21f7c684-2bb5-4c67-9009-1d9f854be081"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:37.302263 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.302227    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-kube-api-access-gttzk" pod "21f7c684-2bb5-4c67-9009-1d9f854be081" (UID: "21f7c684-2bb5-4c67-9009-1d9f854be081"). InnerVolumeSpecName "kube-api-access-gttzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:37.303525 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.303476    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-token" pod "21f7c684-2bb5-4c67-9009-1d9f854be081" (UID: "21f7c684-2bb5-4c67-9009-1d9f854be081"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:37.318060 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.318036    4454 scope.go:122] "RemoveContainer" containerID="ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d"
Nov 03 21:48:37.318761 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:37.318733    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d\": container with ID starting with ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d not found: ID does not exist" containerID="ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d"
Nov 03 21:48:37.318845 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.318774    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d"} err="failed to get container status \"ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d\": rpc error: code = NotFound desc = could not find container \"ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d\": container with ID starting with ea932e61d102b40e93b4f2f6670c8968dd7b6f0721ba1dca7ac2c4082b2aab7d not found: ID does not exist"
Nov 03 21:48:37.318845 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.318799    4454 scope.go:122] "RemoveContainer" containerID="b55135cf5db3afe449733f517d589fd3a5612d789265301840d3e24202895fba"
Nov 03 21:48:37.333533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.333274    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:48:37.335763 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.335741    4454 scope.go:122] "RemoveContainer" containerID="2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545"
Nov 03 21:48:37.347570 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.347543    4454 scope.go:122] "RemoveContainer" containerID="2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545"
Nov 03 21:48:37.348143 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:37.348117    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545\": container with ID starting with 2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545 not found: ID does not exist" containerID="2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545"
Nov 03 21:48:37.348266 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.348154    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545"} err="failed to get container status \"2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545\": rpc error: code = NotFound desc = could not find container \"2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545\": container with ID starting with 2756b50edd811fbd858eec07a8b58bb093204f74ff6607f5d3fff703db25f545 not found: ID does not exist"
Nov 03 21:48:37.357577 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.357249    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"]
Nov 03 21:48:37.371298 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.369823    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-zl2f4"]
Nov 03 21:48:37.395525 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.395465    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-token\" (UniqueName: \"kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-token\") pod \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") "
Nov 03 21:48:37.398000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.397725    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-podinfo\") pod \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") "
Nov 03 21:48:37.398000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.397777    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-data\") pod \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") "
Nov 03 21:48:37.398000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.397615    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-token" pod "2588bcca-4842-455d-a4fd-7397fa0bdb5f" (UID: "2588bcca-4842-455d-a4fd-7397fa0bdb5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:37.398000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.397829    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-kube-api-access-pv2zs\" (UniqueName: \"kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-kube-api-access-pv2zs\") pod \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") "
Nov 03 21:48:37.399352 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.398380    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-envoy\") pod \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") "
Nov 03 21:48:37.399352 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.398481    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-certs\") pod \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") "
Nov 03 21:48:37.399352 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.398646    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istiod-ca-cert\") pod \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") "
Nov 03 21:48:37.399352 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.398807    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-socket\") pod \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") "
Nov 03 21:48:37.399352 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.398850    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-credential-socket\") pod \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\" (UID: \"2588bcca-4842-455d-a4fd-7397fa0bdb5f\") "
Nov 03 21:48:37.399352 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.398974    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-data" pod "2588bcca-4842-455d-a4fd-7397fa0bdb5f" (UID: "2588bcca-4842-455d-a4fd-7397fa0bdb5f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.399352 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399169    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-certs" pod "2588bcca-4842-455d-a4fd-7397fa0bdb5f" (UID: "2588bcca-4842-455d-a4fd-7397fa0bdb5f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.399808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399404    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21f7c684-2bb5-4c67-9009-1d9f854be081-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.399808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399417    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gttzk\" (UniqueName: \"kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-kube-api-access-gttzk\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.399808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399427    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.399808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399435    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.399808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399442    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.399808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399449    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.399808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399456    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.399808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399464    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.399808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399472    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.399808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399480    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.399808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399488    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21f7c684-2bb5-4c67-9009-1d9f854be081-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.399808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399696    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istiod-ca-cert" pod "2588bcca-4842-455d-a4fd-7397fa0bdb5f" (UID: "2588bcca-4842-455d-a4fd-7397fa0bdb5f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:37.400414 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.399844    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-socket" pod "2588bcca-4842-455d-a4fd-7397fa0bdb5f" (UID: "2588bcca-4842-455d-a4fd-7397fa0bdb5f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.400566 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.400538    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-credential-socket" pod "2588bcca-4842-455d-a4fd-7397fa0bdb5f" (UID: "2588bcca-4842-455d-a4fd-7397fa0bdb5f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.401312 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.401285    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-kube-api-access-pv2zs" pod "2588bcca-4842-455d-a4fd-7397fa0bdb5f" (UID: "2588bcca-4842-455d-a4fd-7397fa0bdb5f"). InnerVolumeSpecName "kube-api-access-pv2zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:37.402205 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.402179    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-podinfo" pod "2588bcca-4842-455d-a4fd-7397fa0bdb5f" (UID: "2588bcca-4842-455d-a4fd-7397fa0bdb5f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:37.402899 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.402866    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-envoy" pod "2588bcca-4842-455d-a4fd-7397fa0bdb5f" (UID: "2588bcca-4842-455d-a4fd-7397fa0bdb5f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:37.501478 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.501445    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.501478 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.501477    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv2zs\" (UniqueName: \"kubernetes.io/projected/2588bcca-4842-455d-a4fd-7397fa0bdb5f-kube-api-access-pv2zs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.501705 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.501492    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.501705 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.501530    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2588bcca-4842-455d-a4fd-7397fa0bdb5f-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.501705 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.501541    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.501705 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.501553    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2588bcca-4842-455d-a4fd-7397fa0bdb5f-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:37.665134 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.664961    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"]
Nov 03 21:48:37.668043 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.668004    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-2qm9n"]
Nov 03 21:48:37.732631 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.732598    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"]
Nov 03 21:48:37.733206 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.733175    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l" podUID="b35f8fb8-4b99-48ac-88d1-81975b9c655e" containerName="maas-api" containerID="cri-o://c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635" gracePeriod=30
Nov 03 21:48:37.816502 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:37.816365    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:38.081117 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.081080    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:48:38.155252 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.155166    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75796449-4fc2-4abb-aa29-0435f1464bd6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75796449-4fc2-4abb-aa29-0435f1464bd6-istiod-ca-cert\") pod \"75796449-4fc2-4abb-aa29-0435f1464bd6\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") "
Nov 03 21:48:38.155908 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.155886    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-envoy\") pod \"75796449-4fc2-4abb-aa29-0435f1464bd6\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") "
Nov 03 21:48:38.156518 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.155809    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75796449-4fc2-4abb-aa29-0435f1464bd6-istiod-ca-cert" pod "75796449-4fc2-4abb-aa29-0435f1464bd6" (UID: "75796449-4fc2-4abb-aa29-0435f1464bd6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:38.156846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.156828    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-token\" (UniqueName: \"kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-token\") pod \"75796449-4fc2-4abb-aa29-0435f1464bd6\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") "
Nov 03 21:48:38.157375 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.157358    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-credential-socket\") pod \"75796449-4fc2-4abb-aa29-0435f1464bd6\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") "
Nov 03 21:48:38.157991 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.157790    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-credential-socket" pod "75796449-4fc2-4abb-aa29-0435f1464bd6" (UID: "75796449-4fc2-4abb-aa29-0435f1464bd6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:38.158210 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.158194    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-socket\") pod \"75796449-4fc2-4abb-aa29-0435f1464bd6\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") "
Nov 03 21:48:38.158564 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.158475    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-socket" pod "75796449-4fc2-4abb-aa29-0435f1464bd6" (UID: "75796449-4fc2-4abb-aa29-0435f1464bd6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:38.158766 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.158748    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-data\") pod \"75796449-4fc2-4abb-aa29-0435f1464bd6\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") "
Nov 03 21:48:38.159197 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.159175    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-certs\") pod \"75796449-4fc2-4abb-aa29-0435f1464bd6\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") "
Nov 03 21:48:38.159731 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.159093    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-data" pod "75796449-4fc2-4abb-aa29-0435f1464bd6" (UID: "75796449-4fc2-4abb-aa29-0435f1464bd6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:38.159731 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.159570    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-certs" pod "75796449-4fc2-4abb-aa29-0435f1464bd6" (UID: "75796449-4fc2-4abb-aa29-0435f1464bd6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:38.160004 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.159985    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-kube-api-access-h6lkj\" (UniqueName: \"kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-kube-api-access-h6lkj\") pod \"75796449-4fc2-4abb-aa29-0435f1464bd6\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") "
Nov 03 21:48:38.160769 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.160497    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-podinfo\") pod \"75796449-4fc2-4abb-aa29-0435f1464bd6\" (UID: \"75796449-4fc2-4abb-aa29-0435f1464bd6\") "
Nov 03 21:48:38.161564 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.161437    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.161696 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.161681    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.161785 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.161775    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.161864 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.161855    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.161945 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.161936    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75796449-4fc2-4abb-aa29-0435f1464bd6-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.178671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.164592    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-token" pod "75796449-4fc2-4abb-aa29-0435f1464bd6" (UID: "75796449-4fc2-4abb-aa29-0435f1464bd6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:38.219730 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.219319    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm" podUID="e376c103-f66d-4082-a27a-09bc2c48557a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:48:38.274847 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.274769    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.276941 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.276907    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l" podUID="b35f8fb8-4b99-48ac-88d1-81975b9c655e" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.45:8443/health\": dial tcp 10.133.0.45:8443: connect: connection refused"
Nov 03 21:48:38.287365 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.287332    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-envoy" pod "75796449-4fc2-4abb-aa29-0435f1464bd6" (UID: "75796449-4fc2-4abb-aa29-0435f1464bd6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:38.294349 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.290418    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-kube-api-access-h6lkj" pod "75796449-4fc2-4abb-aa29-0435f1464bd6" (UID: "75796449-4fc2-4abb-aa29-0435f1464bd6"). InnerVolumeSpecName "kube-api-access-h6lkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:38.294349 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.290879    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-podinfo" pod "75796449-4fc2-4abb-aa29-0435f1464bd6" (UID: "75796449-4fc2-4abb-aa29-0435f1464bd6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:38.298783 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.295474    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95" event={"ID":"2588bcca-4842-455d-a4fd-7397fa0bdb5f","Type":"ContainerDied","Data":"c37a0a46464905b77c5c2bb6b1f98facadb71506cd5672d37d560966887ea52f"}
Nov 03 21:48:38.298783 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.295613    4454 scope.go:122] "RemoveContainer" containerID="d774c87dcd02ee26c5b237661769506c1351fda91c42e3863169616aab1bde70"
Nov 03 21:48:38.298783 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.295826    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"
Nov 03 21:48:38.304890 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.300022    4454 generic.go:356] "Generic (PLEG): container finished" podID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerID="f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3" exitCode=0
Nov 03 21:48:38.304890 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.300100    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" event={"ID":"75796449-4fc2-4abb-aa29-0435f1464bd6","Type":"ContainerDied","Data":"f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3"}
Nov 03 21:48:38.304890 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.300141    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22" event={"ID":"75796449-4fc2-4abb-aa29-0435f1464bd6","Type":"ContainerDied","Data":"c898ae86afb2f806c04abe253c8b3ced3e437fc8be3c718c3ed92ba72f1794c7"}
Nov 03 21:48:38.304890 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.300216    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"
Nov 03 21:48:38.332574 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.331861    4454 generic.go:356] "Generic (PLEG): container finished" podID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerID="b22f14209489eb3100c547e4ee76a85724471e18170cf26a73ccf16504afb7d0" exitCode=0
Nov 03 21:48:38.332574 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.332047    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" event={"ID":"b9a8e74a-6b9e-43df-b41e-20ecf8181814","Type":"ContainerDied","Data":"b22f14209489eb3100c547e4ee76a85724471e18170cf26a73ccf16504afb7d0"}
Nov 03 21:48:38.358561 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.355675    4454 scope.go:122] "RemoveContainer" containerID="f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3"
Nov 03 21:48:38.381801 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.381773    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.382000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.381984    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6lkj\" (UniqueName: \"kubernetes.io/projected/75796449-4fc2-4abb-aa29-0435f1464bd6-kube-api-access-h6lkj\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.382103 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.382088    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75796449-4fc2-4abb-aa29-0435f1464bd6-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.388569 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.388502    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"]
Nov 03 21:48:38.390345 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.390323    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-s7b22"]
Nov 03 21:48:38.396243 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.396227    4454 scope.go:122] "RemoveContainer" containerID="f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3"
Nov 03 21:48:38.396598 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:38.396570    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3\": container with ID starting with f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3 not found: ID does not exist" containerID="f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3"
Nov 03 21:48:38.396694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.396608    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3"} err="failed to get container status \"f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3\": rpc error: code = NotFound desc = could not find container \"f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3\": container with ID starting with f63334a5722d6dc5d5be96b0b799b72072af6a00551932b180818bab4f0ae1c3 not found: ID does not exist"
Nov 03 21:48:38.396694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.396632    4454 scope.go:122] "RemoveContainer" containerID="021254eeddaafccc5fd66b10d434f48a69095d210d07373915b313da279876f2"
Nov 03 21:48:38.401598 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.401577    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:48:38.423192 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.423131    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"]
Nov 03 21:48:38.426273 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.426250    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-e246f9-openshift-default-65c46599d9-z8z95"]
Nov 03 21:48:38.498258 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.498222    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:38.583900 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.583866    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-kube-api-access-729ts\" (UniqueName: \"kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-kube-api-access-729ts\") pod \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") "
Nov 03 21:48:38.583900 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.583903    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-podinfo\") pod \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") "
Nov 03 21:48:38.584107 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.583930    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-data\") pod \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") "
Nov 03 21:48:38.584203 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.584180    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-data" pod "b9a8e74a-6b9e-43df-b41e-20ecf8181814" (UID: "b9a8e74a-6b9e-43df-b41e-20ecf8181814"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:38.584203 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.584202    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-envoy\") pod \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") "
Nov 03 21:48:38.584389 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.584276    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-credential-socket\") pod \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") "
Nov 03 21:48:38.584445 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.584388    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-token\" (UniqueName: \"kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-token\") pod \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") "
Nov 03 21:48:38.584590 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.584444    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-socket\") pod \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") "
Nov 03 21:48:38.584590 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.584479    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-certs\") pod \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") "
Nov 03 21:48:38.584590 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.584560    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istiod-ca-cert\") pod \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\" (UID: \"b9a8e74a-6b9e-43df-b41e-20ecf8181814\") "
Nov 03 21:48:38.584590 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.584562    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-credential-socket" pod "b9a8e74a-6b9e-43df-b41e-20ecf8181814" (UID: "b9a8e74a-6b9e-43df-b41e-20ecf8181814"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:38.584805 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.584769    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-socket" pod "b9a8e74a-6b9e-43df-b41e-20ecf8181814" (UID: "b9a8e74a-6b9e-43df-b41e-20ecf8181814"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:38.585129 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.585087    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-certs" pod "b9a8e74a-6b9e-43df-b41e-20ecf8181814" (UID: "b9a8e74a-6b9e-43df-b41e-20ecf8181814"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:38.585129 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.585089    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istiod-ca-cert" pod "b9a8e74a-6b9e-43df-b41e-20ecf8181814" (UID: "b9a8e74a-6b9e-43df-b41e-20ecf8181814"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:38.585256 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.585193    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.585256 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.585214    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.585256 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.585230    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.586335 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.586312    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-envoy" pod "b9a8e74a-6b9e-43df-b41e-20ecf8181814" (UID: "b9a8e74a-6b9e-43df-b41e-20ecf8181814"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:38.586594 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.586574    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-podinfo" pod "b9a8e74a-6b9e-43df-b41e-20ecf8181814" (UID: "b9a8e74a-6b9e-43df-b41e-20ecf8181814"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:38.586666 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.586610    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-kube-api-access-729ts" pod "b9a8e74a-6b9e-43df-b41e-20ecf8181814" (UID: "b9a8e74a-6b9e-43df-b41e-20ecf8181814"). InnerVolumeSpecName "kube-api-access-729ts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:38.586943 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.586908    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-token" pod "b9a8e74a-6b9e-43df-b41e-20ecf8181814" (UID: "b9a8e74a-6b9e-43df-b41e-20ecf8181814"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:38.661597 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.656130    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:38.686298 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.686262    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-729ts\" (UniqueName: \"kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-kube-api-access-729ts\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.686298 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.686293    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.686298 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.686302    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.686298 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.686310    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.686667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.686318    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9a8e74a-6b9e-43df-b41e-20ecf8181814-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.686667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.686326    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9a8e74a-6b9e-43df-b41e-20ecf8181814-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:38.774671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:38.774634    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:39.080151 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.080121    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"
Nov 03 21:48:39.092916 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.092437    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e376c103-f66d-4082-a27a-09bc2c48557a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e376c103-f66d-4082-a27a-09bc2c48557a-plugins-config-volume\") pod \"e376c103-f66d-4082-a27a-09bc2c48557a\" (UID: \"e376c103-f66d-4082-a27a-09bc2c48557a\") "
Nov 03 21:48:39.092916 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.092551    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e376c103-f66d-4082-a27a-09bc2c48557a-kube-api-access-qkrch\" (UniqueName: \"kubernetes.io/projected/e376c103-f66d-4082-a27a-09bc2c48557a-kube-api-access-qkrch\") pod \"e376c103-f66d-4082-a27a-09bc2c48557a\" (UID: \"e376c103-f66d-4082-a27a-09bc2c48557a\") "
Nov 03 21:48:39.092916 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.092846    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e376c103-f66d-4082-a27a-09bc2c48557a-plugins-config-volume" pod "e376c103-f66d-4082-a27a-09bc2c48557a" (UID: "e376c103-f66d-4082-a27a-09bc2c48557a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:39.093467 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.092980    4454 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e376c103-f66d-4082-a27a-09bc2c48557a-plugins-config-volume\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.095783 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.095729    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e376c103-f66d-4082-a27a-09bc2c48557a-kube-api-access-qkrch" pod "e376c103-f66d-4082-a27a-09bc2c48557a" (UID: "e376c103-f66d-4082-a27a-09bc2c48557a"). InnerVolumeSpecName "kube-api-access-qkrch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:39.122741 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.122717    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"
Nov 03 21:48:39.131560 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.131531    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:48:39.172354 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.172262    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:39.194262 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.194228    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-metrics-certs\") pod \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") "
Nov 03 21:48:39.194262 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.194260    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-maas-api-tls\") pod \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") "
Nov 03 21:48:39.194480 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.194337    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b35f8fb8-4b99-48ac-88d1-81975b9c655e-kube-api-access-qxq6j\" (UniqueName: \"kubernetes.io/projected/b35f8fb8-4b99-48ac-88d1-81975b9c655e-kube-api-access-qxq6j\") pod \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") "
Nov 03 21:48:39.194480 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.194398    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b35f8fb8-4b99-48ac-88d1-81975b9c655e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35f8fb8-4b99-48ac-88d1-81975b9c655e-openshift-service-ca\") pod \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\" (UID: \"b35f8fb8-4b99-48ac-88d1-81975b9c655e\") "
Nov 03 21:48:39.194480 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.194418    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-kube-api-access-vsjgz\" (UniqueName: \"kubernetes.io/projected/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-kube-api-access-vsjgz\") pod \"d6ab1abf-4dd9-43d2-845e-225243e8d9d6\" (UID: \"d6ab1abf-4dd9-43d2-845e-225243e8d9d6\") "
Nov 03 21:48:39.194480 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.194436    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-plugins-config-volume\") pod \"d6ab1abf-4dd9-43d2-845e-225243e8d9d6\" (UID: \"d6ab1abf-4dd9-43d2-845e-225243e8d9d6\") "
Nov 03 21:48:39.195203 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.195147    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkrch\" (UniqueName: \"kubernetes.io/projected/e376c103-f66d-4082-a27a-09bc2c48557a-kube-api-access-qkrch\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.195386 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.195355    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b35f8fb8-4b99-48ac-88d1-81975b9c655e-openshift-service-ca" pod "b35f8fb8-4b99-48ac-88d1-81975b9c655e" (UID: "b35f8fb8-4b99-48ac-88d1-81975b9c655e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:39.198706 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.196988    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-plugins-config-volume" pod "d6ab1abf-4dd9-43d2-845e-225243e8d9d6" (UID: "d6ab1abf-4dd9-43d2-845e-225243e8d9d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:39.199143 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.199104    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b35f8fb8-4b99-48ac-88d1-81975b9c655e-kube-api-access-qxq6j" pod "b35f8fb8-4b99-48ac-88d1-81975b9c655e" (UID: "b35f8fb8-4b99-48ac-88d1-81975b9c655e"). InnerVolumeSpecName "kube-api-access-qxq6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:39.199310 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.199258    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-metrics-certs" pod "b35f8fb8-4b99-48ac-88d1-81975b9c655e" (UID: "b35f8fb8-4b99-48ac-88d1-81975b9c655e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:39.200771 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.200749    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-maas-api-tls" pod "b35f8fb8-4b99-48ac-88d1-81975b9c655e" (UID: "b35f8fb8-4b99-48ac-88d1-81975b9c655e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:48:39.201020 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.200993    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-kube-api-access-vsjgz" pod "d6ab1abf-4dd9-43d2-845e-225243e8d9d6" (UID: "d6ab1abf-4dd9-43d2-845e-225243e8d9d6"). InnerVolumeSpecName "kube-api-access-vsjgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:39.231848 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.231817    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" path="/var/lib/kubelet/pods/21f7c684-2bb5-4c67-9009-1d9f854be081/volumes"
Nov 03 21:48:39.232300 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.232287    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" path="/var/lib/kubelet/pods/2588bcca-4842-455d-a4fd-7397fa0bdb5f/volumes"
Nov 03 21:48:39.232715 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.232697    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" path="/var/lib/kubelet/pods/75796449-4fc2-4abb-aa29-0435f1464bd6/volumes"
Nov 03 21:48:39.233434 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.233419    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" path="/var/lib/kubelet/pods/ad15d2cf-954f-4711-a95f-5ce2a24fd12f/volumes"
Nov 03 21:48:39.296271 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.296238    4454 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-metrics-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.296271 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.296267    4454 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b35f8fb8-4b99-48ac-88d1-81975b9c655e-maas-api-tls\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.296271 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.296276    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxq6j\" (UniqueName: \"kubernetes.io/projected/b35f8fb8-4b99-48ac-88d1-81975b9c655e-kube-api-access-qxq6j\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.296480 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.296286    4454 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35f8fb8-4b99-48ac-88d1-81975b9c655e-openshift-service-ca\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.296480 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.296295    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsjgz\" (UniqueName: \"kubernetes.io/projected/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-kube-api-access-vsjgz\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.296480 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.296303    4454 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6ab1abf-4dd9-43d2-845e-225243e8d9d6-plugins-config-volume\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.343600 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.343560    4454 generic.go:356] "Generic (PLEG): container finished" podID="b35f8fb8-4b99-48ac-88d1-81975b9c655e" containerID="c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635" exitCode=0
Nov 03 21:48:39.343765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.343628    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"
Nov 03 21:48:39.343765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.343651    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l" event={"ID":"b35f8fb8-4b99-48ac-88d1-81975b9c655e","Type":"ContainerDied","Data":"c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635"}
Nov 03 21:48:39.343765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.343712    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l" event={"ID":"b35f8fb8-4b99-48ac-88d1-81975b9c655e","Type":"ContainerDied","Data":"a1fe3c1cd934a4dc9e7a795163dba5a304aac78cb1f4d5dae8d9d1301471c1aa"}
Nov 03 21:48:39.343765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.343738    4454 scope.go:122] "RemoveContainer" containerID="c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635"
Nov 03 21:48:39.349265 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.348665    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"
Nov 03 21:48:39.349265 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.348674    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm" event={"ID":"b9a8e74a-6b9e-43df-b41e-20ecf8181814","Type":"ContainerDied","Data":"ccfa0eae63d857ab3ecdc18a33448ae79a6faa6fa149cf1d6740679d8f977956"}
Nov 03 21:48:39.350365 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.350340    4454 generic.go:356] "Generic (PLEG): container finished" podID="e376c103-f66d-4082-a27a-09bc2c48557a" containerID="a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32" exitCode=0
Nov 03 21:48:39.350484 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.350382    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"
Nov 03 21:48:39.350484 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.350471    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm" event={"ID":"e376c103-f66d-4082-a27a-09bc2c48557a","Type":"ContainerDied","Data":"a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32"}
Nov 03 21:48:39.350634 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.350537    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm" event={"ID":"e376c103-f66d-4082-a27a-09bc2c48557a","Type":"ContainerDied","Data":"48f1726d26f926e17e4a0f15eb93e52a11ba3277c84667729a1c8d43b8d7246b"}
Nov 03 21:48:39.352425 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.352300    4454 generic.go:356] "Generic (PLEG): container finished" podID="d6ab1abf-4dd9-43d2-845e-225243e8d9d6" containerID="5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34" exitCode=0
Nov 03 21:48:39.352425 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.352333    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8" event={"ID":"d6ab1abf-4dd9-43d2-845e-225243e8d9d6","Type":"ContainerDied","Data":"5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34"}
Nov 03 21:48:39.352425 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.352394    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8" event={"ID":"d6ab1abf-4dd9-43d2-845e-225243e8d9d6","Type":"ContainerDied","Data":"f72832122f78d44cf7b16404e8f867b04673883c922993c3436194a529548e4e"}
Nov 03 21:48:39.352425 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.352367    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"
Nov 03 21:48:39.359289 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.359268    4454 scope.go:122] "RemoveContainer" containerID="c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635"
Nov 03 21:48:39.359744 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:39.359723    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635\": container with ID starting with c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635 not found: ID does not exist" containerID="c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635"
Nov 03 21:48:39.359874 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.359754    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635"} err="failed to get container status \"c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635\": rpc error: code = NotFound desc = could not find container \"c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635\": container with ID starting with c0a408f9dadebb4cdf3a19f96b381e8584fbb6a839836f32b6dd451470971635 not found: ID does not exist"
Nov 03 21:48:39.359874 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.359777    4454 scope.go:122] "RemoveContainer" containerID="b22f14209489eb3100c547e4ee76a85724471e18170cf26a73ccf16504afb7d0"
Nov 03 21:48:39.369325 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.369308    4454 scope.go:122] "RemoveContainer" containerID="a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32"
Nov 03 21:48:39.377802 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.377784    4454 scope.go:122] "RemoveContainer" containerID="a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32"
Nov 03 21:48:39.378083 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:39.378059    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32\": container with ID starting with a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32 not found: ID does not exist" containerID="a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32"
Nov 03 21:48:39.378163 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.378093    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32"} err="failed to get container status \"a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32\": rpc error: code = NotFound desc = could not find container \"a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32\": container with ID starting with a2977b475e8f4be9a064989177353a55ac2945d9c2ee6dc19868f74e7770ed32 not found: ID does not exist"
Nov 03 21:48:39.378163 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.378111    4454 scope.go:122] "RemoveContainer" containerID="5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34"
Nov 03 21:48:39.386990 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.386970    4454 scope.go:122] "RemoveContainer" containerID="5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34"
Nov 03 21:48:39.389982 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:39.389954    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34\": container with ID starting with 5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34 not found: ID does not exist" containerID="5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34"
Nov 03 21:48:39.390080 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.389990    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34"} err="failed to get container status \"5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34\": rpc error: code = NotFound desc = could not find container \"5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34\": container with ID starting with 5e776b604ea5f29a51484b94b7660403fe15d562fbc08af5defc8abe6f551a34 not found: ID does not exist"
Nov 03 21:48:39.505704 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.505676    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"]
Nov 03 21:48:39.515796 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.511952    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-27fdf4-65947f86bd-qxl7l"]
Nov 03 21:48:39.540615 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.540568    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"]
Nov 03 21:48:39.549642 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.549611    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-0a6da1-openshift-default-5d8fb4b669-wn4tm"]
Nov 03 21:48:39.583548 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.582012    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"]
Nov 03 21:48:39.585667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.585633    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-27fdf4-c7bc87b5f-jsvzm"]
Nov 03 21:48:39.622751 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.622714    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"]
Nov 03 21:48:39.626868 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.626834    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-27fdf4-7bf75d8bc6-qwwr8"]
Nov 03 21:48:39.628417 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.628327    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:39.683554 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.683495    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:48:39.701525 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.701438    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-socket\") pod \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") "
Nov 03 21:48:39.701525 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.701496    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-envoy\") pod \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") "
Nov 03 21:48:39.701712 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.701567    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-credential-socket\") pod \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") "
Nov 03 21:48:39.701712 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.701628    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-podinfo\") pod \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") "
Nov 03 21:48:39.701712 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.701679    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-token\" (UniqueName: \"kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-token\") pod \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") "
Nov 03 21:48:39.701864 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.701759    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-certs\") pod \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") "
Nov 03 21:48:39.701864 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.701806    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-kube-api-access-85mh7\" (UniqueName: \"kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-kube-api-access-85mh7\") pod \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") "
Nov 03 21:48:39.701864 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.701851    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istiod-ca-cert\") pod \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") "
Nov 03 21:48:39.702016 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.701919    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-data\") pod \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\" (UID: \"397ba3eb-ea51-46a3-ad14-b52e3d107db1\") "
Nov 03 21:48:39.702495 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.702470    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-data" pod "397ba3eb-ea51-46a3-ad14-b52e3d107db1" (UID: "397ba3eb-ea51-46a3-ad14-b52e3d107db1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:39.702678 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.702662    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-socket" pod "397ba3eb-ea51-46a3-ad14-b52e3d107db1" (UID: "397ba3eb-ea51-46a3-ad14-b52e3d107db1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:39.704481 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.704206    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-credential-socket" pod "397ba3eb-ea51-46a3-ad14-b52e3d107db1" (UID: "397ba3eb-ea51-46a3-ad14-b52e3d107db1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:39.704856 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.704832    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-envoy" pod "397ba3eb-ea51-46a3-ad14-b52e3d107db1" (UID: "397ba3eb-ea51-46a3-ad14-b52e3d107db1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:39.705551 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.705260    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istiod-ca-cert" pod "397ba3eb-ea51-46a3-ad14-b52e3d107db1" (UID: "397ba3eb-ea51-46a3-ad14-b52e3d107db1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:39.705733 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.705697    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-certs" pod "397ba3eb-ea51-46a3-ad14-b52e3d107db1" (UID: "397ba3eb-ea51-46a3-ad14-b52e3d107db1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:39.706404 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.706380    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-podinfo" pod "397ba3eb-ea51-46a3-ad14-b52e3d107db1" (UID: "397ba3eb-ea51-46a3-ad14-b52e3d107db1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:39.707141 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.707110    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-token" pod "397ba3eb-ea51-46a3-ad14-b52e3d107db1" (UID: "397ba3eb-ea51-46a3-ad14-b52e3d107db1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:39.709035 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.709007    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-kube-api-access-85mh7" pod "397ba3eb-ea51-46a3-ad14-b52e3d107db1" (UID: "397ba3eb-ea51-46a3-ad14-b52e3d107db1"). InnerVolumeSpecName "kube-api-access-85mh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:39.755332 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.755300    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:48:39.803481 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.803448    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-kube-api-access-45nrw\" (UniqueName: \"kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-kube-api-access-45nrw\") pod \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") "
Nov 03 21:48:39.803726 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.803556    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-envoy\") pod \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") "
Nov 03 21:48:39.803726 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.803624    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-socket\") pod \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") "
Nov 03 21:48:39.803726 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.803672    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-token\" (UniqueName: \"kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-token\") pod \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") "
Nov 03 21:48:39.803920 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.803763    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-certs\") pod \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") "
Nov 03 21:48:39.803920 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.803799    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istiod-ca-cert\") pod \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") "
Nov 03 21:48:39.804018 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.803862    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-data\") pod \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") "
Nov 03 21:48:39.804071 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.804038    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-credential-socket\") pod \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") "
Nov 03 21:48:39.804143 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.804081    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-podinfo\") pod \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\" (UID: \"93c5a0bd-9878-4b3b-9f7d-e047754c5c36\") "
Nov 03 21:48:39.804269 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.804232    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-certs" pod "93c5a0bd-9878-4b3b-9f7d-e047754c5c36" (UID: "93c5a0bd-9878-4b3b-9f7d-e047754c5c36"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:39.804689 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.804667    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.804936 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.804920    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.805071 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.805059    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.805166 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.805153    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.805253 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.805240    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.805580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.805314    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-data" pod "93c5a0bd-9878-4b3b-9f7d-e047754c5c36" (UID: "93c5a0bd-9878-4b3b-9f7d-e047754c5c36"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:39.805580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.805331    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.805580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.805347    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85mh7\" (UniqueName: \"kubernetes.io/projected/397ba3eb-ea51-46a3-ad14-b52e3d107db1-kube-api-access-85mh7\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.805580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.805361    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.805580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.805376    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/397ba3eb-ea51-46a3-ad14-b52e3d107db1-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.805580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.805389    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.805580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.805450    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istiod-ca-cert" pod "93c5a0bd-9878-4b3b-9f7d-e047754c5c36" (UID: "93c5a0bd-9878-4b3b-9f7d-e047754c5c36"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:39.805580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.805486    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-credential-socket" pod "93c5a0bd-9878-4b3b-9f7d-e047754c5c36" (UID: "93c5a0bd-9878-4b3b-9f7d-e047754c5c36"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:39.805580 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.805557    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-socket" pod "93c5a0bd-9878-4b3b-9f7d-e047754c5c36" (UID: "93c5a0bd-9878-4b3b-9f7d-e047754c5c36"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:39.807356 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.807315    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-envoy" pod "93c5a0bd-9878-4b3b-9f7d-e047754c5c36" (UID: "93c5a0bd-9878-4b3b-9f7d-e047754c5c36"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:39.807761 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.807730    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-token" pod "93c5a0bd-9878-4b3b-9f7d-e047754c5c36" (UID: "93c5a0bd-9878-4b3b-9f7d-e047754c5c36"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:39.808384 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.808050    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-podinfo" pod "93c5a0bd-9878-4b3b-9f7d-e047754c5c36" (UID: "93c5a0bd-9878-4b3b-9f7d-e047754c5c36"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:39.808637 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.808552    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-kube-api-access-45nrw" pod "93c5a0bd-9878-4b3b-9f7d-e047754c5c36" (UID: "93c5a0bd-9878-4b3b-9f7d-e047754c5c36"). InnerVolumeSpecName "kube-api-access-45nrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:39.906400 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.906363    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.906400 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.906396    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.906400 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.906405    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.906715 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.906415    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.906715 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.906424    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45nrw\" (UniqueName: \"kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-kube-api-access-45nrw\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.906715 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.906434    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.906715 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.906442    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:39.906715 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:39.906449    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c5a0bd-9878-4b3b-9f7d-e047754c5c36-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:40.361421 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.361385    4454 generic.go:356] "Generic (PLEG): container finished" podID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" containerID="f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2" exitCode=0
Nov 03 21:48:40.361892 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.361438    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"
Nov 03 21:48:40.361892 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.361457    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" event={"ID":"397ba3eb-ea51-46a3-ad14-b52e3d107db1","Type":"ContainerDied","Data":"f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2"}
Nov 03 21:48:40.361892 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.361543    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk" event={"ID":"397ba3eb-ea51-46a3-ad14-b52e3d107db1","Type":"ContainerDied","Data":"ca11efb9c6452c59e5b007a3ae09d022656b5933853fceeb62c8947f8798597f"}
Nov 03 21:48:40.361892 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.361572    4454 scope.go:122] "RemoveContainer" containerID="f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2"
Nov 03 21:48:40.364362 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.364337    4454 generic.go:356] "Generic (PLEG): container finished" podID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerID="bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b" exitCode=0
Nov 03 21:48:40.364480 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.364388    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"
Nov 03 21:48:40.364480 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.364415    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" event={"ID":"93c5a0bd-9878-4b3b-9f7d-e047754c5c36","Type":"ContainerDied","Data":"bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b"}
Nov 03 21:48:40.364592 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.364477    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b" event={"ID":"93c5a0bd-9878-4b3b-9f7d-e047754c5c36","Type":"ContainerDied","Data":"28259d7b640d193f3351251d688b652f9846a6c7c11e4e593886ea517236d2cb"}
Nov 03 21:48:40.375355 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.375321    4454 scope.go:122] "RemoveContainer" containerID="f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2"
Nov 03 21:48:40.376083 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:40.376057    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2\": container with ID starting with f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2 not found: ID does not exist" containerID="f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2"
Nov 03 21:48:40.376163 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.376092    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2"} err="failed to get container status \"f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2\": rpc error: code = NotFound desc = could not find container \"f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2\": container with ID starting with f761c91340cc42873c46a6d079fd51d9339affc99561657b9f3a5b7027daa4e2 not found: ID does not exist"
Nov 03 21:48:40.376163 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.376118    4454 scope.go:122] "RemoveContainer" containerID="bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b"
Nov 03 21:48:40.389545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.389501    4454 scope.go:122] "RemoveContainer" containerID="bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b"
Nov 03 21:48:40.389823 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:40.389804    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b\": container with ID starting with bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b not found: ID does not exist" containerID="bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b"
Nov 03 21:48:40.389893 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.389832    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b"} err="failed to get container status \"bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b\": rpc error: code = NotFound desc = could not find container \"bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b\": container with ID starting with bbf7b029a93d4b1a11f59ef096a575c5ea6de85fa2655e44a14e64eff41a831b not found: ID does not exist"
Nov 03 21:48:40.428500 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.428463    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"]
Nov 03 21:48:40.438404 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.438370    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-drfpk"]
Nov 03 21:48:40.464390 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.464358    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"]
Nov 03 21:48:40.470425 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.470396    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-lhf6b"]
Nov 03 21:48:40.766852 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:40.766771    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:41.066748 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:41.066666    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:41.082866 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:41.082832    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:41.229665 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:41.229629    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" path="/var/lib/kubelet/pods/397ba3eb-ea51-46a3-ad14-b52e3d107db1/volumes"
Nov 03 21:48:41.230226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:41.230210    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" path="/var/lib/kubelet/pods/93c5a0bd-9878-4b3b-9f7d-e047754c5c36/volumes"
Nov 03 21:48:41.230628 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:41.230615    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b35f8fb8-4b99-48ac-88d1-81975b9c655e" path="/var/lib/kubelet/pods/b35f8fb8-4b99-48ac-88d1-81975b9c655e/volumes"
Nov 03 21:48:41.231348 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:41.231334    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" path="/var/lib/kubelet/pods/b9a8e74a-6b9e-43df-b41e-20ecf8181814/volumes"
Nov 03 21:48:41.231708 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:41.231695    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6ab1abf-4dd9-43d2-845e-225243e8d9d6" path="/var/lib/kubelet/pods/d6ab1abf-4dd9-43d2-845e-225243e8d9d6/volumes"
Nov 03 21:48:41.232030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:41.232011    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e376c103-f66d-4082-a27a-09bc2c48557a" path="/var/lib/kubelet/pods/e376c103-f66d-4082-a27a-09bc2c48557a/volumes"
Nov 03 21:48:42.528914 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:42.528879    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:42.746735 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:42.746694    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:42.846280 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:42.846248    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:43.104502 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.104431    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:43.153180 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153146    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f4fb28f3-d432-4531-a424-b2063795e50f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4fb28f3-d432-4531-a424-b2063795e50f-istio-podinfo\") pod \"f4fb28f3-d432-4531-a424-b2063795e50f\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") "
Nov 03 21:48:43.153362 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153215    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-certs\") pod \"f4fb28f3-d432-4531-a424-b2063795e50f\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") "
Nov 03 21:48:43.153362 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153248    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4fb28f3-d432-4531-a424-b2063795e50f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4fb28f3-d432-4531-a424-b2063795e50f-istiod-ca-cert\") pod \"f4fb28f3-d432-4531-a424-b2063795e50f\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") "
Nov 03 21:48:43.153362 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153273    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-istio-token\" (UniqueName: \"kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-istio-token\") pod \"f4fb28f3-d432-4531-a424-b2063795e50f\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") "
Nov 03 21:48:43.153362 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153327    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-envoy\") pod \"f4fb28f3-d432-4531-a424-b2063795e50f\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") "
Nov 03 21:48:43.153591 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153372    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-credential-socket\") pod \"f4fb28f3-d432-4531-a424-b2063795e50f\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") "
Nov 03 21:48:43.153591 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153481    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-data\") pod \"f4fb28f3-d432-4531-a424-b2063795e50f\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") "
Nov 03 21:48:43.153591 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153542    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-certs" pod "f4fb28f3-d432-4531-a424-b2063795e50f" (UID: "f4fb28f3-d432-4531-a424-b2063795e50f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:43.153591 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153547    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-kube-api-access-hpm6b\" (UniqueName: \"kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-kube-api-access-hpm6b\") pod \"f4fb28f3-d432-4531-a424-b2063795e50f\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") "
Nov 03 21:48:43.153788 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153665    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4fb28f3-d432-4531-a424-b2063795e50f-istiod-ca-cert" pod "f4fb28f3-d432-4531-a424-b2063795e50f" (UID: "f4fb28f3-d432-4531-a424-b2063795e50f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:43.153788 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153693    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-socket\") pod \"f4fb28f3-d432-4531-a424-b2063795e50f\" (UID: \"f4fb28f3-d432-4531-a424-b2063795e50f\") "
Nov 03 21:48:43.153897 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153835    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-data" pod "f4fb28f3-d432-4531-a424-b2063795e50f" (UID: "f4fb28f3-d432-4531-a424-b2063795e50f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:43.154006 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.153986    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-socket" pod "f4fb28f3-d432-4531-a424-b2063795e50f" (UID: "f4fb28f3-d432-4531-a424-b2063795e50f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:43.154068 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.154013    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-credential-socket" pod "f4fb28f3-d432-4531-a424-b2063795e50f" (UID: "f4fb28f3-d432-4531-a424-b2063795e50f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:43.154292 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.154264    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.154383 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.154297    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.154383 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.154313    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4fb28f3-d432-4531-a424-b2063795e50f-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.154383 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.154326    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.155773 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.155748    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-istio-token" pod "f4fb28f3-d432-4531-a424-b2063795e50f" (UID: "f4fb28f3-d432-4531-a424-b2063795e50f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:43.156021 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.155924    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-envoy" pod "f4fb28f3-d432-4531-a424-b2063795e50f" (UID: "f4fb28f3-d432-4531-a424-b2063795e50f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:43.156021 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.155984    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4fb28f3-d432-4531-a424-b2063795e50f-istio-podinfo" pod "f4fb28f3-d432-4531-a424-b2063795e50f" (UID: "f4fb28f3-d432-4531-a424-b2063795e50f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:43.156289 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.156262    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:43.156462 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.156439    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-kube-api-access-hpm6b" pod "f4fb28f3-d432-4531-a424-b2063795e50f" (UID: "f4fb28f3-d432-4531-a424-b2063795e50f"). InnerVolumeSpecName "kube-api-access-hpm6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:43.255333 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.255306    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.255333 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.255331    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.255333 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.255339    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4fb28f3-d432-4531-a424-b2063795e50f-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.255568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.255350    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpm6b\" (UniqueName: \"kubernetes.io/projected/f4fb28f3-d432-4531-a424-b2063795e50f-kube-api-access-hpm6b\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.255568 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.255360    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4fb28f3-d432-4531-a424-b2063795e50f-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.402058 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.401972    4454 generic.go:356] "Generic (PLEG): container finished" podID="f4fb28f3-d432-4531-a424-b2063795e50f" containerID="ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9" exitCode=0
Nov 03 21:48:43.402212 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.402076    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" event={"ID":"f4fb28f3-d432-4531-a424-b2063795e50f","Type":"ContainerDied","Data":"ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9"}
Nov 03 21:48:43.402212 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.402133    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5" event={"ID":"f4fb28f3-d432-4531-a424-b2063795e50f","Type":"ContainerDied","Data":"065dc3130c9f588489146cdaad4d9f912fd519129d019aa0b987dadb5d32d719"}
Nov 03 21:48:43.402212 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.402165    4454 scope.go:122] "RemoveContainer" containerID="ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9"
Nov 03 21:48:43.402376 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.402282    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"
Nov 03 21:48:43.407543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.404951    4454 generic.go:356] "Generic (PLEG): container finished" podID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerID="fdff37e7c6734683664812b09f993663fb8df038db4f7d29db3107e2fc167f6e" exitCode=0
Nov 03 21:48:43.407543 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.405045    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" event={"ID":"772b7fa0-1b5d-4796-956f-2f5fca140669","Type":"ContainerDied","Data":"fdff37e7c6734683664812b09f993663fb8df038db4f7d29db3107e2fc167f6e"}
Nov 03 21:48:43.414498 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.414482    4454 scope.go:122] "RemoveContainer" containerID="ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9"
Nov 03 21:48:43.414784 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:43.414767    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9\": container with ID starting with ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9 not found: ID does not exist" containerID="ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9"
Nov 03 21:48:43.414830 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.414793    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9"} err="failed to get container status \"ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9\": rpc error: code = NotFound desc = could not find container \"ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9\": container with ID starting with ddb2bdebc957c620086c437f2d2173c70fad0fc5931ed4e49a23562e4d4e56e9 not found: ID does not exist"
Nov 03 21:48:43.415431 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.415415    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:43.452205 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.452170    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"]
Nov 03 21:48:43.460174 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.460149    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/772b7fa0-1b5d-4796-956f-2f5fca140669-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/772b7fa0-1b5d-4796-956f-2f5fca140669-istiod-ca-cert\") pod \"772b7fa0-1b5d-4796-956f-2f5fca140669\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") "
Nov 03 21:48:43.460323 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.460180    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-kube-api-access-7b78g\" (UniqueName: \"kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-kube-api-access-7b78g\") pod \"772b7fa0-1b5d-4796-956f-2f5fca140669\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") "
Nov 03 21:48:43.460323 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.460208    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-podinfo\") pod \"772b7fa0-1b5d-4796-956f-2f5fca140669\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") "
Nov 03 21:48:43.460323 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.460243    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-data\") pod \"772b7fa0-1b5d-4796-956f-2f5fca140669\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") "
Nov 03 21:48:43.460462 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.460339    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-token\" (UniqueName: \"kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-token\") pod \"772b7fa0-1b5d-4796-956f-2f5fca140669\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") "
Nov 03 21:48:43.460462 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.460374    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-certs\") pod \"772b7fa0-1b5d-4796-956f-2f5fca140669\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") "
Nov 03 21:48:43.460647 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.460555    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-socket\") pod \"772b7fa0-1b5d-4796-956f-2f5fca140669\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") "
Nov 03 21:48:43.460647 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.460593    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-envoy\") pod \"772b7fa0-1b5d-4796-956f-2f5fca140669\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") "
Nov 03 21:48:43.460647 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.460590    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/772b7fa0-1b5d-4796-956f-2f5fca140669-istiod-ca-cert" pod "772b7fa0-1b5d-4796-956f-2f5fca140669" (UID: "772b7fa0-1b5d-4796-956f-2f5fca140669"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:43.460647 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.460640    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-credential-socket\") pod \"772b7fa0-1b5d-4796-956f-2f5fca140669\" (UID: \"772b7fa0-1b5d-4796-956f-2f5fca140669\") "
Nov 03 21:48:43.460843 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.460745    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-certs" pod "772b7fa0-1b5d-4796-956f-2f5fca140669" (UID: "772b7fa0-1b5d-4796-956f-2f5fca140669"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:43.460895 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.460853    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-socket" pod "772b7fa0-1b5d-4796-956f-2f5fca140669" (UID: "772b7fa0-1b5d-4796-956f-2f5fca140669"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:43.461102 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.461021    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-credential-socket" pod "772b7fa0-1b5d-4796-956f-2f5fca140669" (UID: "772b7fa0-1b5d-4796-956f-2f5fca140669"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:43.461102 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.461079    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-data" pod "772b7fa0-1b5d-4796-956f-2f5fca140669" (UID: "772b7fa0-1b5d-4796-956f-2f5fca140669"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:43.461342 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.461319    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.461413 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.461361    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.461413 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.461377    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.461413 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.461392    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/772b7fa0-1b5d-4796-956f-2f5fca140669-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.461413 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.461404    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.462724 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.462690    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-envoy" pod "772b7fa0-1b5d-4796-956f-2f5fca140669" (UID: "772b7fa0-1b5d-4796-956f-2f5fca140669"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:43.462846 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.462827    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-podinfo" pod "772b7fa0-1b5d-4796-956f-2f5fca140669" (UID: "772b7fa0-1b5d-4796-956f-2f5fca140669"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:43.463007 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.462986    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-kube-api-access-7b78g" pod "772b7fa0-1b5d-4796-956f-2f5fca140669" (UID: "772b7fa0-1b5d-4796-956f-2f5fca140669"). InnerVolumeSpecName "kube-api-access-7b78g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:43.463825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.463805    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-xs2l5"]
Nov 03 21:48:43.464131 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.464111    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-token" pod "772b7fa0-1b5d-4796-956f-2f5fca140669" (UID: "772b7fa0-1b5d-4796-956f-2f5fca140669"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:43.562871 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.562841    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.562871 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.562866    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.562871 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.562875    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b78g\" (UniqueName: \"kubernetes.io/projected/772b7fa0-1b5d-4796-956f-2f5fca140669-kube-api-access-7b78g\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:43.563285 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:43.562884    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/772b7fa0-1b5d-4796-956f-2f5fca140669-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:44.296827 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:44.296723    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:44.413902 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:44.413877    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"
Nov 03 21:48:44.414074 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:44.413912    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl" event={"ID":"772b7fa0-1b5d-4796-956f-2f5fca140669","Type":"ContainerDied","Data":"e0f0160ef6886886ff05a76da5691262d958ed4a0bfe952431745e73302037e0"}
Nov 03 21:48:44.414074 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:44.413988    4454 scope.go:122] "RemoveContainer" containerID="fdff37e7c6734683664812b09f993663fb8df038db4f7d29db3107e2fc167f6e"
Nov 03 21:48:44.475011 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:44.474985    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"]
Nov 03 21:48:44.485969 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:44.485948    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-05bc27-openshift-default-6df586d6ff-njqvl"]
Nov 03 21:48:44.590015 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:44.589941    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:45.230005 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:45.229977    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" path="/var/lib/kubelet/pods/772b7fa0-1b5d-4796-956f-2f5fca140669/volumes"
Nov 03 21:48:45.230433 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:45.230420    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" path="/var/lib/kubelet/pods/f4fb28f3-d432-4531-a424-b2063795e50f/volumes"
Nov 03 21:48:45.513967 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:45.513875    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:45.616715 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:45.616680    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:45.844361 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:45.844277    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"]
Nov 03 21:48:45.844577 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:45.844533    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy" containerID="cri-o://d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2" gracePeriod=30
Nov 03 21:48:45.857324 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:45.856399    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"]
Nov 03 21:48:45.857324 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:45.856643    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" podUID="5c1f09c7-5b84-4789-9b2b-17657863cd18" containerName="istio-proxy" containerID="cri-o://cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e" gracePeriod=30
Nov 03 21:48:45.859338 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:45.859310    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"]
Nov 03 21:48:45.859545 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:45.859522    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy" containerID="cri-o://8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e" gracePeriod=30
Nov 03 21:48:45.868196 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:45.868172    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:46.310337 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:46.310305    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:46.478890 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:46.478849    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:46.836155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:46.836115    4454 patch_prober.go:29] interesting pod/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Nov 03 21:48:46.836563 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:46.836181    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" podUID="5c1f09c7-5b84-4789-9b2b-17657863cd18" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:48:47.038876 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:47.038843    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:48.114781 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:48.114745    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"]
Nov 03 21:48:48.217603 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:48.217567    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:48.319439 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:48.319365    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:49.916100 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:49.916064    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:50.033111 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:50.032938    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:51.110181 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.110154    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:48:51.160655 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.160630    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:48:51.170000 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.169972    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:48:51.269752 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.269723    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-podinfo\") pod \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") "
Nov 03 21:48:51.269899 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.269789    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c1f09c7-5b84-4789-9b2b-17657863cd18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c1f09c7-5b84-4789-9b2b-17657863cd18-istiod-ca-cert\") pod \"5c1f09c7-5b84-4789-9b2b-17657863cd18\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") "
Nov 03 21:48:51.269899 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.269858    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-credential-socket\") pod \"e8126051-d2b3-488f-86d0-a2919329a4db\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") "
Nov 03 21:48:51.270005 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.269908    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-data\") pod \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") "
Nov 03 21:48:51.270005 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.269973    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-kube-api-access-vnd5r\" (UniqueName: \"kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-kube-api-access-vnd5r\") pod \"e8126051-d2b3-488f-86d0-a2919329a4db\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") "
Nov 03 21:48:51.270132 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270011    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-kube-api-access-bm4nz\" (UniqueName: \"kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-kube-api-access-bm4nz\") pod \"5c1f09c7-5b84-4789-9b2b-17657863cd18\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") "
Nov 03 21:48:51.270132 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270064    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e8126051-d2b3-488f-86d0-a2919329a4db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8126051-d2b3-488f-86d0-a2919329a4db-istio-podinfo\") pod \"e8126051-d2b3-488f-86d0-a2919329a4db\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") "
Nov 03 21:48:51.270132 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270096    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-data\") pod \"e8126051-d2b3-488f-86d0-a2919329a4db\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") "
Nov 03 21:48:51.270132 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270128    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-envoy\") pod \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270155    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-credential-socket\") pod \"5c1f09c7-5b84-4789-9b2b-17657863cd18\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270158    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-credential-socket" pod "e8126051-d2b3-488f-86d0-a2919329a4db" (UID: "e8126051-d2b3-488f-86d0-a2919329a4db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270174    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c1f09c7-5b84-4789-9b2b-17657863cd18-istiod-ca-cert" pod "5c1f09c7-5b84-4789-9b2b-17657863cd18" (UID: "5c1f09c7-5b84-4789-9b2b-17657863cd18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270194    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-certs\") pod \"5c1f09c7-5b84-4789-9b2b-17657863cd18\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270224    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-podinfo\") pod \"5c1f09c7-5b84-4789-9b2b-17657863cd18\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270253    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-socket\") pod \"5c1f09c7-5b84-4789-9b2b-17657863cd18\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270271    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-data" pod "b32c16e9-42d8-4d67-85d2-42826a1ff1b1" (UID: "b32c16e9-42d8-4d67-85d2-42826a1ff1b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270282    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-credential-socket\") pod \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270316    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8126051-d2b3-488f-86d0-a2919329a4db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8126051-d2b3-488f-86d0-a2919329a4db-istiod-ca-cert\") pod \"e8126051-d2b3-488f-86d0-a2919329a4db\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270347    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-data\") pod \"5c1f09c7-5b84-4789-9b2b-17657863cd18\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270374    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-certs\") pod \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270410    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-socket\") pod \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270446    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-envoy\") pod \"5c1f09c7-5b84-4789-9b2b-17657863cd18\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270487    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-data" pod "e8126051-d2b3-488f-86d0-a2919329a4db" (UID: "e8126051-d2b3-488f-86d0-a2919329a4db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270498    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-socket\") pod \"e8126051-d2b3-488f-86d0-a2919329a4db\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270556    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-istio-token\" (UniqueName: \"kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-istio-token\") pod \"e8126051-d2b3-488f-86d0-a2919329a4db\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270588    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-token\" (UniqueName: \"kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-token\") pod \"5c1f09c7-5b84-4789-9b2b-17657863cd18\" (UID: \"5c1f09c7-5b84-4789-9b2b-17657863cd18\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270616    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istiod-ca-cert\") pod \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270649    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-envoy\") pod \"e8126051-d2b3-488f-86d0-a2919329a4db\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270682    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-kube-api-access-9lbqd\" (UniqueName: \"kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-kube-api-access-9lbqd\") pod \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270656    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-credential-socket" pod "b32c16e9-42d8-4d67-85d2-42826a1ff1b1" (UID: "b32c16e9-42d8-4d67-85d2-42826a1ff1b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270718    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-certs" pod "b32c16e9-42d8-4d67-85d2-42826a1ff1b1" (UID: "b32c16e9-42d8-4d67-85d2-42826a1ff1b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270722    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-token\" (UniqueName: \"kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-token\") pod \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\" (UID: \"b32c16e9-42d8-4d67-85d2-42826a1ff1b1\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270779    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-certs\") pod \"e8126051-d2b3-488f-86d0-a2919329a4db\" (UID: \"e8126051-d2b3-488f-86d0-a2919329a4db\") "
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.270907    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-credential-socket" pod "5c1f09c7-5b84-4789-9b2b-17657863cd18" (UID: "5c1f09c7-5b84-4789-9b2b-17657863cd18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.271301    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c1f09c7-5b84-4789-9b2b-17657863cd18-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.271322    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.271336    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.271350    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.271339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.271364    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.272847 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.271378    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.272847 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.271393    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.272847 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.271589    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-certs" pod "e8126051-d2b3-488f-86d0-a2919329a4db" (UID: "e8126051-d2b3-488f-86d0-a2919329a4db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.272847 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.271855    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8126051-d2b3-488f-86d0-a2919329a4db-istiod-ca-cert" pod "e8126051-d2b3-488f-86d0-a2919329a4db" (UID: "e8126051-d2b3-488f-86d0-a2919329a4db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:51.272847 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.272087    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-data" pod "5c1f09c7-5b84-4789-9b2b-17657863cd18" (UID: "5c1f09c7-5b84-4789-9b2b-17657863cd18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.272847 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.272435    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-podinfo" pod "b32c16e9-42d8-4d67-85d2-42826a1ff1b1" (UID: "b32c16e9-42d8-4d67-85d2-42826a1ff1b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:51.273816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.273231    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-certs" pod "5c1f09c7-5b84-4789-9b2b-17657863cd18" (UID: "5c1f09c7-5b84-4789-9b2b-17657863cd18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.273816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.273458    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-socket" pod "5c1f09c7-5b84-4789-9b2b-17657863cd18" (UID: "5c1f09c7-5b84-4789-9b2b-17657863cd18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.273816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.273563    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-kube-api-access-bm4nz" pod "5c1f09c7-5b84-4789-9b2b-17657863cd18" (UID: "5c1f09c7-5b84-4789-9b2b-17657863cd18"). InnerVolumeSpecName "kube-api-access-bm4nz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:51.274452 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.274427    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8126051-d2b3-488f-86d0-a2919329a4db-istio-podinfo" pod "e8126051-d2b3-488f-86d0-a2919329a4db" (UID: "e8126051-d2b3-488f-86d0-a2919329a4db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:51.274812 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.274561    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-token" pod "b32c16e9-42d8-4d67-85d2-42826a1ff1b1" (UID: "b32c16e9-42d8-4d67-85d2-42826a1ff1b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:51.274812 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.274660    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-socket" pod "e8126051-d2b3-488f-86d0-a2919329a4db" (UID: "e8126051-d2b3-488f-86d0-a2919329a4db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.274812 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.274666    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-podinfo" pod "5c1f09c7-5b84-4789-9b2b-17657863cd18" (UID: "5c1f09c7-5b84-4789-9b2b-17657863cd18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:48:51.274812 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.274706    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-envoy" pod "b32c16e9-42d8-4d67-85d2-42826a1ff1b1" (UID: "b32c16e9-42d8-4d67-85d2-42826a1ff1b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.275294 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.274920    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istiod-ca-cert" pod "b32c16e9-42d8-4d67-85d2-42826a1ff1b1" (UID: "b32c16e9-42d8-4d67-85d2-42826a1ff1b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:48:51.275294 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.275134    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-socket" pod "b32c16e9-42d8-4d67-85d2-42826a1ff1b1" (UID: "b32c16e9-42d8-4d67-85d2-42826a1ff1b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.275294 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.275220    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-istio-token" pod "e8126051-d2b3-488f-86d0-a2919329a4db" (UID: "e8126051-d2b3-488f-86d0-a2919329a4db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:51.275294 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.275282    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-kube-api-access-vnd5r" pod "e8126051-d2b3-488f-86d0-a2919329a4db" (UID: "e8126051-d2b3-488f-86d0-a2919329a4db"). InnerVolumeSpecName "kube-api-access-vnd5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:51.276326 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.276304    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-token" pod "5c1f09c7-5b84-4789-9b2b-17657863cd18" (UID: "5c1f09c7-5b84-4789-9b2b-17657863cd18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:51.276833 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.276808    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-envoy" pod "5c1f09c7-5b84-4789-9b2b-17657863cd18" (UID: "5c1f09c7-5b84-4789-9b2b-17657863cd18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.277250 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.277222    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-envoy" pod "e8126051-d2b3-488f-86d0-a2919329a4db" (UID: "e8126051-d2b3-488f-86d0-a2919329a4db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:48:51.277570 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.277494    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-kube-api-access-9lbqd" pod "b32c16e9-42d8-4d67-85d2-42826a1ff1b1" (UID: "b32c16e9-42d8-4d67-85d2-42826a1ff1b1"). InnerVolumeSpecName "kube-api-access-9lbqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:48:51.372556 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372532    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8126051-d2b3-488f-86d0-a2919329a4db-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.372662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372560    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.372662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372572    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.372662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372584    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.372662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372599    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.372662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372610    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.372662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372623    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.372662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372635    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.372662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372646    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.372662 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372656    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lbqd\" (UniqueName: \"kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-kube-api-access-9lbqd\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.373077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372670    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.373077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372684    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8126051-d2b3-488f-86d0-a2919329a4db-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.373077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372696    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.373077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372709    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnd5r\" (UniqueName: \"kubernetes.io/projected/e8126051-d2b3-488f-86d0-a2919329a4db-kube-api-access-vnd5r\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.373077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372723    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bm4nz\" (UniqueName: \"kubernetes.io/projected/5c1f09c7-5b84-4789-9b2b-17657863cd18-kube-api-access-bm4nz\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.373077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372736    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8126051-d2b3-488f-86d0-a2919329a4db-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.373077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372752    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b32c16e9-42d8-4d67-85d2-42826a1ff1b1-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.373077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372765    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.373077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372777    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c1f09c7-5b84-4789-9b2b-17657863cd18-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.373077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.372790    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1f09c7-5b84-4789-9b2b-17657863cd18-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:48:51.458823 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.458786    4454 generic.go:356] "Generic (PLEG): container finished" podID="e8126051-d2b3-488f-86d0-a2919329a4db" containerID="d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2" exitCode=0
Nov 03 21:48:51.458906 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.458825    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"
Nov 03 21:48:51.458906 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.458838    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" event={"ID":"e8126051-d2b3-488f-86d0-a2919329a4db","Type":"ContainerDied","Data":"d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2"}
Nov 03 21:48:51.458906 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.458895    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k" event={"ID":"e8126051-d2b3-488f-86d0-a2919329a4db","Type":"ContainerDied","Data":"d41bcdc16a31180f8a583476e0088eb6193600a981623e8174f59d79c1d7bd7d"}
Nov 03 21:48:51.459070 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.458922    4454 scope.go:122] "RemoveContainer" containerID="d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2"
Nov 03 21:48:51.460442 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.460413    4454 generic.go:356] "Generic (PLEG): container finished" podID="5c1f09c7-5b84-4789-9b2b-17657863cd18" containerID="cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e" exitCode=0
Nov 03 21:48:51.460573 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.460467    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"
Nov 03 21:48:51.460573 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.460551    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" event={"ID":"5c1f09c7-5b84-4789-9b2b-17657863cd18","Type":"ContainerDied","Data":"cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e"}
Nov 03 21:48:51.460692 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.460595    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q" event={"ID":"5c1f09c7-5b84-4789-9b2b-17657863cd18","Type":"ContainerDied","Data":"b22ca4aee32f465bbdaa6321b4e3729d00d4294b283b63aa1aa0f52f3f46b7d9"}
Nov 03 21:48:51.462684 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.462660    4454 generic.go:356] "Generic (PLEG): container finished" podID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerID="8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e" exitCode=0
Nov 03 21:48:51.462784 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.462701    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"
Nov 03 21:48:51.462835 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.462823    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" event={"ID":"b32c16e9-42d8-4d67-85d2-42826a1ff1b1","Type":"ContainerDied","Data":"8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e"}
Nov 03 21:48:51.462878 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.462861    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9" event={"ID":"b32c16e9-42d8-4d67-85d2-42826a1ff1b1","Type":"ContainerDied","Data":"9c6d95c115389619fcb4cc5276ae43a80d979d068e86dc09ba4a7c527a9e8fd2"}
Nov 03 21:48:51.476201 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.476184    4454 scope.go:122] "RemoveContainer" containerID="d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2"
Nov 03 21:48:51.476471 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:51.476453    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2\": container with ID starting with d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2 not found: ID does not exist" containerID="d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2"
Nov 03 21:48:51.476552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.476484    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2"} err="failed to get container status \"d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2\": rpc error: code = NotFound desc = could not find container \"d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2\": container with ID starting with d8707a151d933c2da8c5df84eb991dd539356cc0dc9ef6e19731c71c604eddb2 not found: ID does not exist"
Nov 03 21:48:51.476552 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.476544    4454 scope.go:122] "RemoveContainer" containerID="cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e"
Nov 03 21:48:51.486951 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.486935    4454 scope.go:122] "RemoveContainer" containerID="cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e"
Nov 03 21:48:51.487206 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:51.487186    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e\": container with ID starting with cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e not found: ID does not exist" containerID="cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e"
Nov 03 21:48:51.487257 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.487217    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e"} err="failed to get container status \"cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e\": rpc error: code = NotFound desc = could not find container \"cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e\": container with ID starting with cd86539af42879a1203237bdade956a5d630b23229ec4a0452df75f05c75ef9e not found: ID does not exist"
Nov 03 21:48:51.487257 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.487234    4454 scope.go:122] "RemoveContainer" containerID="8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e"
Nov 03 21:48:51.494742 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.494726    4454 scope.go:122] "RemoveContainer" containerID="8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e"
Nov 03 21:48:51.494992 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:51.494975    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e\": container with ID starting with 8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e not found: ID does not exist" containerID="8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e"
Nov 03 21:48:51.495033 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.494999    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e"} err="failed to get container status \"8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e\": rpc error: code = NotFound desc = could not find container \"8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e\": container with ID starting with 8b817912fcca8503d5c2c33e9c67cb0bab9c53a52475518d9d1cf6254f08b63e not found: ID does not exist"
Nov 03 21:48:51.533094 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.533069    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"]
Nov 03 21:48:51.543604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.543582    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-c7d6q"]
Nov 03 21:48:51.564801 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.564772    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"]
Nov 03 21:48:51.570612 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.570580    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-vxst9"]
Nov 03 21:48:51.601064 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.601040    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"]
Nov 03 21:48:51.604827 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.604794    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-27fdf4-openshift-default-67c598bf99-t5k9k"]
Nov 03 21:48:51.767724 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:51.767629    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:52.370459 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:48:52.370423    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:48:53.229604 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:53.229573    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c1f09c7-5b84-4789-9b2b-17657863cd18" path="/var/lib/kubelet/pods/5c1f09c7-5b84-4789-9b2b-17657863cd18/volumes"
Nov 03 21:48:53.230018 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:53.230004    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" path="/var/lib/kubelet/pods/b32c16e9-42d8-4d67-85d2-42826a1ff1b1/volumes"
Nov 03 21:48:53.230391 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:53.230380    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" path="/var/lib/kubelet/pods/e8126051-d2b3-488f-86d0-a2919329a4db/volumes"
Nov 03 21:48:53.465820 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:53.465783    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:53.869912 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:53.869870    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:48:53.974318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:48:53.974283    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:00.820297 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:00.820260    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:03.135664 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.135601    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" containerID="cri-o://8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158" gracePeriod=45
Nov 03 21:49:03.387024 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.386965    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:49:03.502421 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.502383    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-model-cache\") pod \"6c4344c2-f323-4ed4-b879-faf1437d6f58\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") "
Nov 03 21:49:03.504155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.502497    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-tmp-dir\") pod \"6c4344c2-f323-4ed4-b879-faf1437d6f58\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") "
Nov 03 21:49:03.504155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.502574    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-dshm\") pod \"6c4344c2-f323-4ed4-b879-faf1437d6f58\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") "
Nov 03 21:49:03.504155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.502652    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-home\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-home\") pod \"6c4344c2-f323-4ed4-b879-faf1437d6f58\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") "
Nov 03 21:49:03.504155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.502732    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c4344c2-f323-4ed4-b879-faf1437d6f58-kube-api-access-xhwlz\" (UniqueName: \"kubernetes.io/projected/6c4344c2-f323-4ed4-b879-faf1437d6f58-kube-api-access-xhwlz\") pod \"6c4344c2-f323-4ed4-b879-faf1437d6f58\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") "
Nov 03 21:49:03.504155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.502795    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6c4344c2-f323-4ed4-b879-faf1437d6f58-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c4344c2-f323-4ed4-b879-faf1437d6f58-tls-certs\") pod \"6c4344c2-f323-4ed4-b879-faf1437d6f58\" (UID: \"6c4344c2-f323-4ed4-b879-faf1437d6f58\") "
Nov 03 21:49:03.504155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.502881    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-model-cache" pod "6c4344c2-f323-4ed4-b879-faf1437d6f58" (UID: "6c4344c2-f323-4ed4-b879-faf1437d6f58"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:03.504155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.503163    4454 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-model-cache\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:03.504155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.503362    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-home" pod "6c4344c2-f323-4ed4-b879-faf1437d6f58" (UID: "6c4344c2-f323-4ed4-b879-faf1437d6f58"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:03.504155 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.503522    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-tmp-dir" pod "6c4344c2-f323-4ed4-b879-faf1437d6f58" (UID: "6c4344c2-f323-4ed4-b879-faf1437d6f58"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:03.506004 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.505965    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c4344c2-f323-4ed4-b879-faf1437d6f58-tls-certs" pod "6c4344c2-f323-4ed4-b879-faf1437d6f58" (UID: "6c4344c2-f323-4ed4-b879-faf1437d6f58"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:49:03.509966 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.509934    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c4344c2-f323-4ed4-b879-faf1437d6f58-kube-api-access-xhwlz" pod "6c4344c2-f323-4ed4-b879-faf1437d6f58" (UID: "6c4344c2-f323-4ed4-b879-faf1437d6f58"). InnerVolumeSpecName "kube-api-access-xhwlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:49:03.510269 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.509992    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-dshm" pod "6c4344c2-f323-4ed4-b879-faf1437d6f58" (UID: "6c4344c2-f323-4ed4-b879-faf1437d6f58"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:03.522202 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.522174    4454 generic.go:356] "Generic (PLEG): container finished" podID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerID="8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158" exitCode=0
Nov 03 21:49:03.522322 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.522227    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"
Nov 03 21:49:03.522322 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.522255    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" event={"ID":"6c4344c2-f323-4ed4-b879-faf1437d6f58","Type":"ContainerDied","Data":"8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158"}
Nov 03 21:49:03.522322 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.522307    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" event={"ID":"6c4344c2-f323-4ed4-b879-faf1437d6f58","Type":"ContainerDied","Data":"c85304f2a6cc46e83f82e7e9754dcd2f78ee8133c12974fe803624e07489499b"}
Nov 03 21:49:03.522462 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.522324    4454 scope.go:122] "RemoveContainer" containerID="8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158"
Nov 03 21:49:03.531902 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.531877    4454 scope.go:122] "RemoveContainer" containerID="8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158"
Nov 03 21:49:03.532198 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:49:03.532172    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158\": container with ID starting with 8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158 not found: ID does not exist" containerID="8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158"
Nov 03 21:49:03.532295 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.532209    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158"} err="failed to get container status \"8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158\": rpc error: code = NotFound desc = could not find container \"8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158\": container with ID starting with 8fbebc37ec466314977b325da565cabadb18e1dfa95930755cd3a04b3c433158 not found: ID does not exist"
Nov 03 21:49:03.571218 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.571157    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"]
Nov 03 21:49:03.574651 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.574628    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm"]
Nov 03 21:49:03.604164 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.604134    4454 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-home\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:03.604164 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.604163    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhwlz\" (UniqueName: \"kubernetes.io/projected/6c4344c2-f323-4ed4-b879-faf1437d6f58-kube-api-access-xhwlz\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:03.604341 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.604173    4454 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c4344c2-f323-4ed4-b879-faf1437d6f58-tls-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:03.604341 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.604182    4454 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-tmp-dir\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:03.604341 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:03.604190    4454 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c4344c2-f323-4ed4-b879-faf1437d6f58-dshm\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:04.000437 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:04.000403    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:49:04.343488 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:04.343361    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-517149/distinct-w6-517149-kserve-6f88746b98-gv2zm" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:49:05.230594 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:05.230555    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" path="/var/lib/kubelet/pods/6c4344c2-f323-4ed4-b879-faf1437d6f58/volumes"
Nov 03 21:49:10.073532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:10.069893    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"]
Nov 03 21:49:10.074394 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:10.074332    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4" podUID="0a1497ee-7ded-402f-b081-9b8a3f7a8825" containerName="maas-api" containerID="cri-o://9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c" gracePeriod=30
Nov 03 21:49:10.202531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:10.199160    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:10.737286 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:10.737248    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:10.835044 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:10.835014    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:11.357847 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.357820    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:49:11.367909 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.367882    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:11.421369 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.421337    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a1497ee-7ded-402f-b081-9b8a3f7a8825-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1497ee-7ded-402f-b081-9b8a3f7a8825-openshift-service-ca\") pod \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") "
Nov 03 21:49:11.421565 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.421382    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-metrics-certs\") pod \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") "
Nov 03 21:49:11.421565 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.421484    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-maas-api-tls\") pod \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") "
Nov 03 21:49:11.421779 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.421752    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a1497ee-7ded-402f-b081-9b8a3f7a8825-openshift-service-ca" pod "0a1497ee-7ded-402f-b081-9b8a3f7a8825" (UID: "0a1497ee-7ded-402f-b081-9b8a3f7a8825"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:49:11.421850 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.421837    4454 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1497ee-7ded-402f-b081-9b8a3f7a8825-openshift-service-ca\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:11.423422 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.423393    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-metrics-certs" pod "0a1497ee-7ded-402f-b081-9b8a3f7a8825" (UID: "0a1497ee-7ded-402f-b081-9b8a3f7a8825"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:49:11.423577 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.423486    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-maas-api-tls" pod "0a1497ee-7ded-402f-b081-9b8a3f7a8825" (UID: "0a1497ee-7ded-402f-b081-9b8a3f7a8825"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:49:11.522929 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.522895    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a1497ee-7ded-402f-b081-9b8a3f7a8825-kube-api-access-frk9w\" (UniqueName: \"kubernetes.io/projected/0a1497ee-7ded-402f-b081-9b8a3f7a8825-kube-api-access-frk9w\") pod \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\" (UID: \"0a1497ee-7ded-402f-b081-9b8a3f7a8825\") "
Nov 03 21:49:11.523277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.523259    4454 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-maas-api-tls\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:11.523277 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.523275    4454 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a1497ee-7ded-402f-b081-9b8a3f7a8825-metrics-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:11.524902 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.524880    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a1497ee-7ded-402f-b081-9b8a3f7a8825-kube-api-access-frk9w" pod "0a1497ee-7ded-402f-b081-9b8a3f7a8825" (UID: "0a1497ee-7ded-402f-b081-9b8a3f7a8825"). InnerVolumeSpecName "kube-api-access-frk9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:49:11.624016 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.623986    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frk9w\" (UniqueName: \"kubernetes.io/projected/0a1497ee-7ded-402f-b081-9b8a3f7a8825-kube-api-access-frk9w\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:11.651689 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.651660    4454 generic.go:356] "Generic (PLEG): container finished" podID="0a1497ee-7ded-402f-b081-9b8a3f7a8825" containerID="9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c" exitCode=0
Nov 03 21:49:11.651816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.651706    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"
Nov 03 21:49:11.651816 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.651708    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4" event={"ID":"0a1497ee-7ded-402f-b081-9b8a3f7a8825","Type":"ContainerDied","Data":"9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c"}
Nov 03 21:49:11.651892 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.651827    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4" event={"ID":"0a1497ee-7ded-402f-b081-9b8a3f7a8825","Type":"ContainerDied","Data":"07f066367b9840950ac15de5ce89c74815941a0810a758c465ba939e1fb91446"}
Nov 03 21:49:11.651892 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.651854    4454 scope.go:122] "RemoveContainer" containerID="9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c"
Nov 03 21:49:11.661318 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.661305    4454 scope.go:122] "RemoveContainer" containerID="9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c"
Nov 03 21:49:11.661593 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:49:11.661562    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c\": container with ID starting with 9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c not found: ID does not exist" containerID="9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c"
Nov 03 21:49:11.661652 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.661604    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c"} err="failed to get container status \"9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c\": rpc error: code = NotFound desc = could not find container \"9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c\": container with ID starting with 9207983e922d82ef054feb568328ed38b65c565a1eeb2b992557b43a56aecb4c not found: ID does not exist"
Nov 03 21:49:11.723595 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.723499    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"]
Nov 03 21:49:11.733384 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:11.733358    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-517149-645559575d-f5tl4"]
Nov 03 21:49:13.229016 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:13.228977    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a1497ee-7ded-402f-b081-9b8a3f7a8825" path="/var/lib/kubelet/pods/0a1497ee-7ded-402f-b081-9b8a3f7a8825/volumes"
Nov 03 21:49:13.456693 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:13.456659    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:49:17.088425 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:17.088385    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:17.189820 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:17.189782    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:17.404680 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:17.404646    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"]
Nov 03 21:49:17.405080 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:17.405049    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk" podUID="bd15fe5f-6e59-4763-af37-ddd5493e1af8" containerName="istio-proxy" containerID="cri-o://5d165d969de9a28eedd053969446c78aca3fb88e4f9db36c9bf926b779c8ffa6" gracePeriod=30
Nov 03 21:49:17.417059 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:17.417027    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"]
Nov 03 21:49:17.417379 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:17.417347    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy" containerID="cri-o://c4f29fa756b548bbb7340e39d7f3e2b539d1a88d86a30dc5b05c4c9a390ca992" gracePeriod=30
Nov 03 21:49:17.420390 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:17.420364    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"]
Nov 03 21:49:17.420792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:17.420766    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerName="istio-proxy" containerID="cri-o://6150896afac42fe08e1f666392c6fa92fa243494143208a32f04855fe326acf2" gracePeriod=30
Nov 03 21:49:17.428612 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:17.428587    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"]
Nov 03 21:49:17.428794 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:17.428774    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7" podUID="32ce613f-1f5c-40f8-a895-ce9dbc0016e4" containerName="istio-proxy" containerID="cri-o://221a3679e751c502fb01fce2e87429a8505cd5f36fa5cf835e1912469e8ff624" gracePeriod=30
Nov 03 21:49:17.469043 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:17.469012    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:21.379620 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:21.379583    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:49:22.380879 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:49:22.380835    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:49:22.726446 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.722397    4454 generic.go:356] "Generic (PLEG): container finished" podID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerID="c4f29fa756b548bbb7340e39d7f3e2b539d1a88d86a30dc5b05c4c9a390ca992" exitCode=0
Nov 03 21:49:22.726446 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.722574    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" event={"ID":"c23ca699-38c0-44f3-bd17-5bc17dcf1963","Type":"ContainerDied","Data":"c4f29fa756b548bbb7340e39d7f3e2b539d1a88d86a30dc5b05c4c9a390ca992"}
Nov 03 21:49:22.726446 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.724955    4454 generic.go:356] "Generic (PLEG): container finished" podID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerID="6150896afac42fe08e1f666392c6fa92fa243494143208a32f04855fe326acf2" exitCode=0
Nov 03 21:49:22.726446 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.725074    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" event={"ID":"a09566e5-58cc-4bb7-bf49-9912a8555677","Type":"ContainerDied","Data":"6150896afac42fe08e1f666392c6fa92fa243494143208a32f04855fe326acf2"}
Nov 03 21:49:22.734678 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.734587    4454 generic.go:356] "Generic (PLEG): container finished" podID="32ce613f-1f5c-40f8-a895-ce9dbc0016e4" containerID="221a3679e751c502fb01fce2e87429a8505cd5f36fa5cf835e1912469e8ff624" exitCode=0
Nov 03 21:49:22.734817 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.734696    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7" event={"ID":"32ce613f-1f5c-40f8-a895-ce9dbc0016e4","Type":"ContainerDied","Data":"221a3679e751c502fb01fce2e87429a8505cd5f36fa5cf835e1912469e8ff624"}
Nov 03 21:49:22.738191 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.736747    4454 generic.go:356] "Generic (PLEG): container finished" podID="bd15fe5f-6e59-4763-af37-ddd5493e1af8" containerID="5d165d969de9a28eedd053969446c78aca3fb88e4f9db36c9bf926b779c8ffa6" exitCode=0
Nov 03 21:49:22.738191 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.736801    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk" event={"ID":"bd15fe5f-6e59-4763-af37-ddd5493e1af8","Type":"ContainerDied","Data":"5d165d969de9a28eedd053969446c78aca3fb88e4f9db36c9bf926b779c8ffa6"}
Nov 03 21:49:22.849120 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.848925    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:49:22.855226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.854738    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.948819    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-credential-socket\") pod \"a09566e5-58cc-4bb7-bf49-9912a8555677\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.948904    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-data\") pod \"a09566e5-58cc-4bb7-bf49-9912a8555677\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.948942    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-data\") pod \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.948976    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a09566e5-58cc-4bb7-bf49-9912a8555677-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a09566e5-58cc-4bb7-bf49-9912a8555677-istiod-ca-cert\") pod \"a09566e5-58cc-4bb7-bf49-9912a8555677\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949005    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-kube-api-access-l5nqr\" (UniqueName: \"kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-kube-api-access-l5nqr\") pod \"a09566e5-58cc-4bb7-bf49-9912a8555677\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949038    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-envoy\") pod \"a09566e5-58cc-4bb7-bf49-9912a8555677\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949072    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-podinfo\") pod \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949113    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-socket\") pod \"a09566e5-58cc-4bb7-bf49-9912a8555677\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949142    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-podinfo\") pod \"a09566e5-58cc-4bb7-bf49-9912a8555677\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949189    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-credential-socket\") pod \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949217    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-certs\") pod \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949248    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-kube-api-access-b7jxf\" (UniqueName: \"kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-kube-api-access-b7jxf\") pod \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949308    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-certs\") pod \"a09566e5-58cc-4bb7-bf49-9912a8555677\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949345    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-socket\") pod \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949380    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-envoy\") pod \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949423    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-token\" (UniqueName: \"kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-token\") pod \"a09566e5-58cc-4bb7-bf49-9912a8555677\" (UID: \"a09566e5-58cc-4bb7-bf49-9912a8555677\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949460    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-token\" (UniqueName: \"kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-token\") pod \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") "
Nov 03 21:49:22.950052 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.949498    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istiod-ca-cert\") pod \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\" (UID: \"32ce613f-1f5c-40f8-a895-ce9dbc0016e4\") "
Nov 03 21:49:22.956961 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.954275    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istiod-ca-cert" pod "32ce613f-1f5c-40f8-a895-ce9dbc0016e4" (UID: "32ce613f-1f5c-40f8-a895-ce9dbc0016e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:49:22.957150 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.957112    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-credential-socket" pod "32ce613f-1f5c-40f8-a895-ce9dbc0016e4" (UID: "32ce613f-1f5c-40f8-a895-ce9dbc0016e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:22.957379 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.957357    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-certs" pod "32ce613f-1f5c-40f8-a895-ce9dbc0016e4" (UID: "32ce613f-1f5c-40f8-a895-ce9dbc0016e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:22.957598 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.957574    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a09566e5-58cc-4bb7-bf49-9912a8555677-istiod-ca-cert" pod "a09566e5-58cc-4bb7-bf49-9912a8555677" (UID: "a09566e5-58cc-4bb7-bf49-9912a8555677"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:49:22.957858 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.957836    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-credential-socket" pod "a09566e5-58cc-4bb7-bf49-9912a8555677" (UID: "a09566e5-58cc-4bb7-bf49-9912a8555677"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:22.958046 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.958025    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-data" pod "a09566e5-58cc-4bb7-bf49-9912a8555677" (UID: "a09566e5-58cc-4bb7-bf49-9912a8555677"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:22.958211 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.958192    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-data" pod "32ce613f-1f5c-40f8-a895-ce9dbc0016e4" (UID: "32ce613f-1f5c-40f8-a895-ce9dbc0016e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:22.998530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.995070    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-certs" pod "a09566e5-58cc-4bb7-bf49-9912a8555677" (UID: "a09566e5-58cc-4bb7-bf49-9912a8555677"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:22.998530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:22.995280    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-socket" pod "32ce613f-1f5c-40f8-a895-ce9dbc0016e4" (UID: "32ce613f-1f5c-40f8-a895-ce9dbc0016e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.036550 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.033626    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-socket" pod "a09566e5-58cc-4bb7-bf49-9912a8555677" (UID: "a09566e5-58cc-4bb7-bf49-9912a8555677"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.052194 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.040634    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-podinfo" pod "a09566e5-58cc-4bb7-bf49-9912a8555677" (UID: "a09566e5-58cc-4bb7-bf49-9912a8555677"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:49:23.052194 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.046854    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-kube-api-access-l5nqr" pod "a09566e5-58cc-4bb7-bf49-9912a8555677" (UID: "a09566e5-58cc-4bb7-bf49-9912a8555677"). InnerVolumeSpecName "kube-api-access-l5nqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:49:23.052194 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.047643    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-kube-api-access-b7jxf" pod "32ce613f-1f5c-40f8-a895-ce9dbc0016e4" (UID: "32ce613f-1f5c-40f8-a895-ce9dbc0016e4"). InnerVolumeSpecName "kube-api-access-b7jxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:49:23.052194 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.048281    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-token" pod "a09566e5-58cc-4bb7-bf49-9912a8555677" (UID: "a09566e5-58cc-4bb7-bf49-9912a8555677"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:49:23.052194 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.048453    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-envoy" pod "a09566e5-58cc-4bb7-bf49-9912a8555677" (UID: "a09566e5-58cc-4bb7-bf49-9912a8555677"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.053756 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.052845    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-token" pod "32ce613f-1f5c-40f8-a895-ce9dbc0016e4" (UID: "32ce613f-1f5c-40f8-a895-ce9dbc0016e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:49:23.053756 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.053541    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.053756 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.053567    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.053756 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.053582    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.053756 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.053595    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.053756 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.053608    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.053756 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.053622    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.053756 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.053635    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.054226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.053958    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.054226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.053978    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a09566e5-58cc-4bb7-bf49-9912a8555677-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.054226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.053991    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5nqr\" (UniqueName: \"kubernetes.io/projected/a09566e5-58cc-4bb7-bf49-9912a8555677-kube-api-access-l5nqr\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.054226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.054006    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.054226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.054020    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a09566e5-58cc-4bb7-bf49-9912a8555677-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.054226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.054033    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a09566e5-58cc-4bb7-bf49-9912a8555677-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.054226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.054046    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.054226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.054059    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.054226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.054073    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7jxf\" (UniqueName: \"kubernetes.io/projected/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-kube-api-access-b7jxf\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.055644 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.055619    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-podinfo" pod "32ce613f-1f5c-40f8-a895-ce9dbc0016e4" (UID: "32ce613f-1f5c-40f8-a895-ce9dbc0016e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:49:23.056933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.056869    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-envoy" pod "32ce613f-1f5c-40f8-a895-ce9dbc0016e4" (UID: "32ce613f-1f5c-40f8-a895-ce9dbc0016e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.081540 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.072677    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:49:23.139123 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.138711    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.154794    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-socket\") pod \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.154866    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-kube-api-access-mhlvz\" (UniqueName: \"kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-kube-api-access-mhlvz\") pod \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.154899    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-credential-socket\") pod \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.154963    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-podinfo\") pod \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.154995    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-kube-api-access-fz5nv\" (UniqueName: \"kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-kube-api-access-fz5nv\") pod \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155021    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-envoy\") pod \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155056    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-certs\") pod \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155091    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istiod-ca-cert\") pod \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155120    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-credential-socket\") pod \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155151    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-token\" (UniqueName: \"kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-token\") pod \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155197    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-certs\") pod \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155227    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-data\") pod \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155255    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-socket\") pod \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155323    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istiod-ca-cert\") pod \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155357    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-envoy\") pod \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155387    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-token\" (UniqueName: \"kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-token\") pod \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155420    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-podinfo\") pod \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\" (UID: \"c23ca699-38c0-44f3-bd17-5bc17dcf1963\") "
Nov 03 21:49:23.155765 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155476    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-data\") pod \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\" (UID: \"bd15fe5f-6e59-4763-af37-ddd5493e1af8\") "
Nov 03 21:49:23.157293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155890    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.157293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155913    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32ce613f-1f5c-40f8-a895-ce9dbc0016e4-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.157293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155223    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-credential-socket" pod "bd15fe5f-6e59-4763-af37-ddd5493e1af8" (UID: "bd15fe5f-6e59-4763-af37-ddd5493e1af8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.157293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155479    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istiod-ca-cert" pod "c23ca699-38c0-44f3-bd17-5bc17dcf1963" (UID: "c23ca699-38c0-44f3-bd17-5bc17dcf1963"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:49:23.157293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155579    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-data" pod "c23ca699-38c0-44f3-bd17-5bc17dcf1963" (UID: "c23ca699-38c0-44f3-bd17-5bc17dcf1963"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.157293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155674    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-certs" pod "c23ca699-38c0-44f3-bd17-5bc17dcf1963" (UID: "c23ca699-38c0-44f3-bd17-5bc17dcf1963"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.157293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.155745    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-credential-socket" pod "c23ca699-38c0-44f3-bd17-5bc17dcf1963" (UID: "c23ca699-38c0-44f3-bd17-5bc17dcf1963"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.157293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.156095    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-data" pod "bd15fe5f-6e59-4763-af37-ddd5493e1af8" (UID: "bd15fe5f-6e59-4763-af37-ddd5493e1af8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.157293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.156355    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-socket" pod "bd15fe5f-6e59-4763-af37-ddd5493e1af8" (UID: "bd15fe5f-6e59-4763-af37-ddd5493e1af8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.157293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.156414    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-certs" pod "bd15fe5f-6e59-4763-af37-ddd5493e1af8" (UID: "bd15fe5f-6e59-4763-af37-ddd5493e1af8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.157293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.156724    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-socket" pod "c23ca699-38c0-44f3-bd17-5bc17dcf1963" (UID: "c23ca699-38c0-44f3-bd17-5bc17dcf1963"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.157293 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.157128    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istiod-ca-cert" pod "bd15fe5f-6e59-4763-af37-ddd5493e1af8" (UID: "bd15fe5f-6e59-4763-af37-ddd5493e1af8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:49:23.158014 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.157675    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-kube-api-access-mhlvz" pod "bd15fe5f-6e59-4763-af37-ddd5493e1af8" (UID: "bd15fe5f-6e59-4763-af37-ddd5493e1af8"). InnerVolumeSpecName "kube-api-access-mhlvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:49:23.158014 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.157740    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-kube-api-access-fz5nv" pod "c23ca699-38c0-44f3-bd17-5bc17dcf1963" (UID: "c23ca699-38c0-44f3-bd17-5bc17dcf1963"). InnerVolumeSpecName "kube-api-access-fz5nv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:49:23.158014 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.157885    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-envoy" pod "bd15fe5f-6e59-4763-af37-ddd5493e1af8" (UID: "bd15fe5f-6e59-4763-af37-ddd5493e1af8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.158167 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.158138    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-podinfo" pod "bd15fe5f-6e59-4763-af37-ddd5493e1af8" (UID: "bd15fe5f-6e59-4763-af37-ddd5493e1af8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:49:23.158735 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.158707    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-token" pod "bd15fe5f-6e59-4763-af37-ddd5493e1af8" (UID: "bd15fe5f-6e59-4763-af37-ddd5493e1af8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:49:23.158938 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.158916    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-podinfo" pod "c23ca699-38c0-44f3-bd17-5bc17dcf1963" (UID: "c23ca699-38c0-44f3-bd17-5bc17dcf1963"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:49:23.158993 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.158921    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-envoy" pod "c23ca699-38c0-44f3-bd17-5bc17dcf1963" (UID: "c23ca699-38c0-44f3-bd17-5bc17dcf1963"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:49:23.159479 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.159459    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-token" pod "c23ca699-38c0-44f3-bd17-5bc17dcf1963" (UID: "c23ca699-38c0-44f3-bd17-5bc17dcf1963"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307588    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fz5nv\" (UniqueName: \"kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-kube-api-access-fz5nv\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307632    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307650    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307673    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307688    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307704    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307720    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307740    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307755    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307771    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307786    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307805    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307820    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c23ca699-38c0-44f3-bd17-5bc17dcf1963-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307834    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307849    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307868    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhlvz\" (UniqueName: \"kubernetes.io/projected/bd15fe5f-6e59-4763-af37-ddd5493e1af8-kube-api-access-mhlvz\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307885    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd15fe5f-6e59-4763-af37-ddd5493e1af8-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.340694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.307900    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd15fe5f-6e59-4763-af37-ddd5493e1af8-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:49:23.747182 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.747152    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"
Nov 03 21:49:23.747723 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.747190    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc" event={"ID":"a09566e5-58cc-4bb7-bf49-9912a8555677","Type":"ContainerDied","Data":"9cf7d73a8885d59ae7d568042e5412fbf6a2eeaf95ae0cebe24561648f59132b"}
Nov 03 21:49:23.747723 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.747369    4454 scope.go:122] "RemoveContainer" containerID="6150896afac42fe08e1f666392c6fa92fa243494143208a32f04855fe326acf2"
Nov 03 21:49:23.750185 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.750151    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7" event={"ID":"32ce613f-1f5c-40f8-a895-ce9dbc0016e4","Type":"ContainerDied","Data":"290a96be2fc8f7cc7cfe3cb3f60a2bcd9750084b7d50821338c9a8f121dd4dc5"}
Nov 03 21:49:23.751208 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.750879    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"
Nov 03 21:49:23.751825 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.751800    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk" event={"ID":"bd15fe5f-6e59-4763-af37-ddd5493e1af8","Type":"ContainerDied","Data":"73595dff7289f117d245ba4fdaa8912053bc6e761a65d2ae91d9d399f67d3106"}
Nov 03 21:49:23.751919 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.751910    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"
Nov 03 21:49:23.754386 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.754362    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts" event={"ID":"c23ca699-38c0-44f3-bd17-5bc17dcf1963","Type":"ContainerDied","Data":"60c327f787f67c6f573539476d20352f99f53fb02d0e7096d729bfe537fb9907"}
Nov 03 21:49:23.754386 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.754382    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"
Nov 03 21:49:23.757226 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.757176    4454 scope.go:122] "RemoveContainer" containerID="221a3679e751c502fb01fce2e87429a8505cd5f36fa5cf835e1912469e8ff624"
Nov 03 21:49:23.785582 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.785556    4454 scope.go:122] "RemoveContainer" containerID="5d165d969de9a28eedd053969446c78aca3fb88e4f9db36c9bf926b779c8ffa6"
Nov 03 21:49:23.840105 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.840074    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"]
Nov 03 21:49:23.842219 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.841699    4454 scope.go:122] "RemoveContainer" containerID="c4f29fa756b548bbb7340e39d7f3e2b539d1a88d86a30dc5b05c4c9a390ca992"
Nov 03 21:49:23.849037 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.849011    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-8vjts"]
Nov 03 21:49:23.890531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.883556    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"]
Nov 03 21:49:23.890531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.884371    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-b9tfc"]
Nov 03 21:49:23.928530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.927554    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"]
Nov 03 21:49:23.928700 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.928631    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-zclhk"]
Nov 03 21:49:23.956009 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.955971    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"]
Nov 03 21:49:23.958154 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:23.958101    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-517149-openshift-default-5cfb5c6655-ttxh7"]
Nov 03 21:49:24.195530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:24.194609    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lq9lw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:49:24.195530 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:24.194674    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw" podUID="6174dd8c-e88b-48eb-8ba6-9935eb711810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:49:24.225532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:24.224629    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x4f2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:49:24.225532 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:24.224694    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g" podUID="ea097b76-14e5-48e9-9966-ed40754d66f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:49:25.230194 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:25.230155    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32ce613f-1f5c-40f8-a895-ce9dbc0016e4" path="/var/lib/kubelet/pods/32ce613f-1f5c-40f8-a895-ce9dbc0016e4/volumes"
Nov 03 21:49:25.230798 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:25.230779    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" path="/var/lib/kubelet/pods/a09566e5-58cc-4bb7-bf49-9912a8555677/volumes"
Nov 03 21:49:25.231237 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:25.231218    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd15fe5f-6e59-4763-af37-ddd5493e1af8" path="/var/lib/kubelet/pods/bd15fe5f-6e59-4763-af37-ddd5493e1af8/volumes"
Nov 03 21:49:25.235589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:25.232243    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" path="/var/lib/kubelet/pods/c23ca699-38c0-44f3-bd17-5bc17dcf1963/volumes"
Nov 03 21:49:25.743121 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:25.743081    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:25.839727 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:25.839691    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:25.932889 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:25.932854    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:31.311533 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:31.311477    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:36.961058 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:36.961018    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:45.372122 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:45.372091    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:49:51.336312 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:51.336280    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:49:51.336948 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:51.336931    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:49:52.369972 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:49:52.369929    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:49:55.065141 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:55.065103    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:57.362409 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:57.362377    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:49:57.918969 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:49:57.918933    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:50:22.370341 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:50:22.370294    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:50:35.272188 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:50:35.272155    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:50:37.187160 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:50:37.187125    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:50:52.357135 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:50:52.357099    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:50:52.370675 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:50:52.370643    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:50:57.081163 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:50:57.081127    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:51:12.564705 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:51:12.564655    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:51:12.672500 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:51:12.672466    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:51:15.575623 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:51:15.575590    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:51:22.370172 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:51:22.370137    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:51:31.655407 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:51:31.655369    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:51:37.289929 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:51:37.289850    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:51:41.667667 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:51:41.667622    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:51:52.369692 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:51:52.369657    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:52:00.664686 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:52:00.664653    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:52:11.092845 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:52:11.092810    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:52:15.569342 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:52:15.569304    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:52:22.369187 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:52:22.369148    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:52:30.101003 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:52:30.100964    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:52:35.565990 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:52:35.565945    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:52:37.406271 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:52:37.406231    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:52:40.345567 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:52:40.345524    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:52:40.445438 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:52:40.445397    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:52:41.017077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:52:41.017030    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:52:52.369950 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:52:52.369912    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:52:56.352742 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:52:56.352703    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:53:00.760563 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:53:00.760532    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:53:19.493850 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:53:19.493802    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:53:22.376946 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:53:22.376901    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:53:24.214531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:53:24.213620    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x4f2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:53:24.214531 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:53:24.213691    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g" podUID="ea097b76-14e5-48e9-9966-ed40754d66f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:53:25.006770 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:53:25.006738    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:53:37.551961 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:53:37.551922    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:53:39.131893 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:53:39.131853    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:53:39.553464 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:53:39.553363    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:53:50.249306 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:53:50.249270    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:53:52.370360 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:53:52.370323    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:54:01.163358 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:54:01.163318    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:54:07.279425 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:54:07.279389    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:54:11.156624 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:54:11.156580    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:54:22.369975 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:54:22.369931    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:54:24.479267 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:54:24.479231    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:54:29.890423 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:54:29.890381    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:54:37.662431 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:54:37.662348    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:54:40.201063 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:54:40.201030    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:54:48.770418 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:54:48.770376    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:54:49.370457 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:54:49.370422    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:54:51.362123 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:54:51.362097    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:54:51.365740 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:54:51.365717    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:54:52.370416 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:54:52.370374    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:54:58.083891 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:54:58.083854    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:55:08.462406 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:08.462372    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:55:22.370713 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:55:22.370672    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:55:28.476220 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:28.476181    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:55:34.271631 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:34.271597    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:55:37.826302 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:37.826263    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:37.925598 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:37.925564    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:38.371345 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:38.371306    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:38.471741 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:38.471688    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:39.081275 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:39.081241    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:42.076694 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:42.076638    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:42.192062 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:42.192011    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:43.719055 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719024    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"]
Nov 03 21:55:43.719841 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719798    4454 cpu_manager.go:414] "containerMap: removing container" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy"
Nov 03 21:55:43.719841 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719827    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy"
Nov 03 21:55:43.719841 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719844    4454 cpu_manager.go:414] "containerMap: removing container" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerName="istio-proxy"
Nov 03 21:55:43.719841 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719854    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719867    4454 cpu_manager.go:414] "containerMap: removing container" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719874    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719887    4454 cpu_manager.go:414] "containerMap: removing container" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719893    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719903    4454 cpu_manager.go:414] "containerMap: removing container" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719909    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719921    4454 cpu_manager.go:414] "containerMap: removing container" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719927    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719937    4454 cpu_manager.go:414] "containerMap: removing container" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719943    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719956    4454 cpu_manager.go:414] "containerMap: removing container" podUID="bd15fe5f-6e59-4763-af37-ddd5493e1af8" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719963    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd15fe5f-6e59-4763-af37-ddd5493e1af8" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719979    4454 cpu_manager.go:414] "containerMap: removing container" podUID="35187015-8165-4126-8832-adcad32ca770" containerName="maas-api"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719984    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35187015-8165-4126-8832-adcad32ca770" containerName="maas-api"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719990    4454 cpu_manager.go:414] "containerMap: removing container" podUID="0a1497ee-7ded-402f-b081-9b8a3f7a8825" containerName="maas-api"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.719994    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a1497ee-7ded-402f-b081-9b8a3f7a8825" containerName="maas-api"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720000    4454 cpu_manager.go:414] "containerMap: removing container" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720004    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720013    4454 cpu_manager.go:414] "containerMap: removing container" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720017    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720023    4454 cpu_manager.go:414] "containerMap: removing container" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720027    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720034    4454 cpu_manager.go:414] "containerMap: removing container" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720039    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720048    4454 cpu_manager.go:414] "containerMap: removing container" podUID="32ce613f-1f5c-40f8-a895-ce9dbc0016e4" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720052    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32ce613f-1f5c-40f8-a895-ce9dbc0016e4" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720059    4454 cpu_manager.go:414] "containerMap: removing container" podUID="e376c103-f66d-4082-a27a-09bc2c48557a" containerName="payload-pre-processing"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720064    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e376c103-f66d-4082-a27a-09bc2c48557a" containerName="payload-pre-processing"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720071    4454 cpu_manager.go:414] "containerMap: removing container" podUID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720075    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720081    4454 cpu_manager.go:414] "containerMap: removing container" podUID="3d31fd0d-3343-45cb-aa88-297d86d9ddfb" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720085    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d31fd0d-3343-45cb-aa88-297d86d9ddfb" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720097    4454 cpu_manager.go:414] "containerMap: removing container" podUID="d6ab1abf-4dd9-43d2-845e-225243e8d9d6" containerName="payload-processing"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720101    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6ab1abf-4dd9-43d2-845e-225243e8d9d6" containerName="payload-processing"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720107    4454 cpu_manager.go:414] "containerMap: removing container" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720111    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720116    4454 cpu_manager.go:414] "containerMap: removing container" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720121    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720130    4454 cpu_manager.go:414] "containerMap: removing container" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy"
Nov 03 21:55:43.720113 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720134    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720140    4454 cpu_manager.go:414] "containerMap: removing container" podUID="b35f8fb8-4b99-48ac-88d1-81975b9c655e" containerName="maas-api"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720144    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b35f8fb8-4b99-48ac-88d1-81975b9c655e" containerName="maas-api"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720150    4454 cpu_manager.go:414] "containerMap: removing container" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720154    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720160    4454 cpu_manager.go:414] "containerMap: removing container" podUID="5c1f09c7-5b84-4789-9b2b-17657863cd18" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720166    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c1f09c7-5b84-4789-9b2b-17657863cd18" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720173    4454 cpu_manager.go:414] "containerMap: removing container" podUID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720177    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720183    4454 cpu_manager.go:414] "containerMap: removing container" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720187    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720195    4454 cpu_manager.go:414] "containerMap: removing container" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720199    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720273    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720279    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad15d2cf-954f-4711-a95f-5ce2a24fd12f" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720285    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd15fe5f-6e59-4763-af37-ddd5493e1af8" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720291    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="35187015-8165-4126-8832-adcad32ca770" containerName="maas-api"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720296    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a1497ee-7ded-402f-b081-9b8a3f7a8825" containerName="maas-api"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720300    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="772b7fa0-1b5d-4796-956f-2f5fca140669" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720308    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4fb28f3-d432-4531-a424-b2063795e50f" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720314    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c4344c2-f323-4ed4-b879-faf1437d6f58" containerName="main"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720318    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="75796449-4fc2-4abb-aa29-0435f1464bd6" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720323    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="32ce613f-1f5c-40f8-a895-ce9dbc0016e4" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720328    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="e376c103-f66d-4082-a27a-09bc2c48557a" containerName="payload-pre-processing"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720332    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="397ba3eb-ea51-46a3-ad14-b52e3d107db1" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720337    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d31fd0d-3343-45cb-aa88-297d86d9ddfb" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720343    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6ab1abf-4dd9-43d2-845e-225243e8d9d6" containerName="payload-processing"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720348    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720352    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720357    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9a8e74a-6b9e-43df-b41e-20ecf8181814" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720361    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="b35f8fb8-4b99-48ac-88d1-81975b9c655e" containerName="maas-api"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720365    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="5eda1eb4-b61c-44fd-ad6c-d11d122f36ad" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720370    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc0ea14b-d4d9-4ab9-bfbe-aa260f33a515" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720375    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c1f09c7-5b84-4789-9b2b-17657863cd18" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720380    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="21f7c684-2bb5-4c67-9009-1d9f854be081" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720385    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="a09566e5-58cc-4bb7-bf49-9912a8555677" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720390    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="2588bcca-4842-455d-a4fd-7397fa0bdb5f" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720394    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="c23ca699-38c0-44f3-bd17-5bc17dcf1963" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720399    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8126051-d2b3-488f-86d0-a2919329a4db" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720403    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="b32c16e9-42d8-4d67-85d2-42826a1ff1b1" containerName="istio-proxy"
Nov 03 21:55:43.721241 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.720408    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="93c5a0bd-9878-4b3b-9f7d-e047754c5c36" containerName="istio-proxy"
Nov 03 21:55:43.723640 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.723618    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.750405 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.750375    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"]
Nov 03 21:55:43.767817 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.767786    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-metrics-certs\") pod \"maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.767980 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.767833    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.767980 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.767856    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt9gn\" (UniqueName: \"kubernetes.io/projected/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-kube-api-access-xt9gn\") pod \"maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.768108 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.768025    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.808009 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.807964    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-9f9b50-serving-cert\""
Nov 03 21:55:43.812916 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.812884    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:43.868797 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.868765    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.868980 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.868827    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-metrics-certs\") pod \"maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.868980 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.868863    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.868980 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.868892    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt9gn\" (UniqueName: \"kubernetes.io/projected/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-kube-api-access-xt9gn\") pod \"maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.869844 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.869820    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.871624 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.871602    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.872059 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.872037    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-metrics-certs\") pod \"maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.889929 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.889902    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt9gn\" (UniqueName: \"kubernetes.io/projected/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-kube-api-access-xt9gn\") pod \"maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:43.912550 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.912476    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:43.968478 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.968448    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"]
Nov 03 21:55:43.978768 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.978699    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"
Nov 03 21:55:43.998086 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.998054    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"]
Nov 03 21:55:43.999559 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:43.999535    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-9f9b50\""
Nov 03 21:55:44.007573 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.007545    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-6b6e8388-dockercfg-p7w8l\""
Nov 03 21:55:44.025948 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.025918    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"]
Nov 03 21:55:44.028581 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.028545    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:44.029461 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.029442    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"
Nov 03 21:55:44.035855 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.035835    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:44.050074 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.050049    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"]
Nov 03 21:55:44.078073 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.072368    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6254ce84-75c8-40d3-a75b-9df6b08aa18c-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm\" (UID: \"6254ce84-75c8-40d3-a75b-9df6b08aa18c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"
Nov 03 21:55:44.078073 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.072489    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdjr9\" (UniqueName: \"kubernetes.io/projected/6254ce84-75c8-40d3-a75b-9df6b08aa18c-kube-api-access-kdjr9\") pod \"payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm\" (UID: \"6254ce84-75c8-40d3-a75b-9df6b08aa18c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"
Nov 03 21:55:44.078073 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.072532    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdad9cf6-50bd-45e4-87ea-7886d27273f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs\" (UID: \"fdad9cf6-50bd-45e4-87ea-7886d27273f2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"
Nov 03 21:55:44.078073 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.072733    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s68nh\" (UniqueName: \"kubernetes.io/projected/fdad9cf6-50bd-45e4-87ea-7886d27273f2-kube-api-access-s68nh\") pod \"payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs\" (UID: \"fdad9cf6-50bd-45e4-87ea-7886d27273f2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"
Nov 03 21:55:44.173347 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.173319    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6254ce84-75c8-40d3-a75b-9df6b08aa18c-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm\" (UID: \"6254ce84-75c8-40d3-a75b-9df6b08aa18c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"
Nov 03 21:55:44.173555 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.173356    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdjr9\" (UniqueName: \"kubernetes.io/projected/6254ce84-75c8-40d3-a75b-9df6b08aa18c-kube-api-access-kdjr9\") pod \"payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm\" (UID: \"6254ce84-75c8-40d3-a75b-9df6b08aa18c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"
Nov 03 21:55:44.173555 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.173375    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdad9cf6-50bd-45e4-87ea-7886d27273f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs\" (UID: \"fdad9cf6-50bd-45e4-87ea-7886d27273f2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"
Nov 03 21:55:44.173555 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.173451    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s68nh\" (UniqueName: \"kubernetes.io/projected/fdad9cf6-50bd-45e4-87ea-7886d27273f2-kube-api-access-s68nh\") pod \"payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs\" (UID: \"fdad9cf6-50bd-45e4-87ea-7886d27273f2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"
Nov 03 21:55:44.174124 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.174103    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6254ce84-75c8-40d3-a75b-9df6b08aa18c-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm\" (UID: \"6254ce84-75c8-40d3-a75b-9df6b08aa18c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"
Nov 03 21:55:44.174252 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.174234    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdad9cf6-50bd-45e4-87ea-7886d27273f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs\" (UID: \"fdad9cf6-50bd-45e4-87ea-7886d27273f2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"
Nov 03 21:55:44.191865 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.191840    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdjr9\" (UniqueName: \"kubernetes.io/projected/6254ce84-75c8-40d3-a75b-9df6b08aa18c-kube-api-access-kdjr9\") pod \"payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm\" (UID: \"6254ce84-75c8-40d3-a75b-9df6b08aa18c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"
Nov 03 21:55:44.192323 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.192300    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s68nh\" (UniqueName: \"kubernetes.io/projected/fdad9cf6-50bd-45e4-87ea-7886d27273f2-kube-api-access-s68nh\") pod \"payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs\" (UID: \"fdad9cf6-50bd-45e4-87ea-7886d27273f2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"
Nov 03 21:55:44.292374 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.292296    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"
Nov 03 21:55:44.342701 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.342479    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"
Nov 03 21:55:44.360852 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.360823    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"]
Nov 03 21:55:44.366866 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:55:44.366841    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa9e821_efa5_4e33_a6a1_02aa18ccf284.slice/crio-38e2ebe61baf69d4a7a16a9466dedd39bac3b313cb28fb1701e54b83bdcdaca9 WatchSource:0}: Error finding container 38e2ebe61baf69d4a7a16a9466dedd39bac3b313cb28fb1701e54b83bdcdaca9: Status 404 returned error can't find the container with id 38e2ebe61baf69d4a7a16a9466dedd39bac3b313cb28fb1701e54b83bdcdaca9
Nov 03 21:55:44.370805 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.370779    4454 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:55:44.582534 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.582449    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"]
Nov 03 21:55:44.587084 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:55:44.587059    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6254ce84_75c8_40d3_a75b_9df6b08aa18c.slice/crio-61d0d3cf582dc3783026cc96054e161e517bc541201c1fb3c67a2658e98f3f3f WatchSource:0}: Error finding container 61d0d3cf582dc3783026cc96054e161e517bc541201c1fb3c67a2658e98f3f3f: Status 404 returned error can't find the container with id 61d0d3cf582dc3783026cc96054e161e517bc541201c1fb3c67a2658e98f3f3f
Nov 03 21:55:44.646488 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.646460    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"]
Nov 03 21:55:44.653039 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:55:44.653010    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdad9cf6_50bd_45e4_87ea_7886d27273f2.slice/crio-8d4fcf8fab211b79db26bc6073bff51116cbc79505b163dcec8a37ba63fbd950 WatchSource:0}: Error finding container 8d4fcf8fab211b79db26bc6073bff51116cbc79505b163dcec8a37ba63fbd950: Status 404 returned error can't find the container with id 8d4fcf8fab211b79db26bc6073bff51116cbc79505b163dcec8a37ba63fbd950
Nov 03 21:55:44.653923 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.653898    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm" event={"ID":"6254ce84-75c8-40d3-a75b-9df6b08aa18c","Type":"ContainerStarted","Data":"61d0d3cf582dc3783026cc96054e161e517bc541201c1fb3c67a2658e98f3f3f"}
Nov 03 21:55:44.655077 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:44.655048    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb" event={"ID":"0aa9e821-efa5-4e33-a6a1-02aa18ccf284","Type":"ContainerStarted","Data":"38e2ebe61baf69d4a7a16a9466dedd39bac3b313cb28fb1701e54b83bdcdaca9"}
Nov 03 21:55:45.660657 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:45.660623    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb" event={"ID":"0aa9e821-efa5-4e33-a6a1-02aa18ccf284","Type":"ContainerStarted","Data":"1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436"}
Nov 03 21:55:45.661164 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:45.660795    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:45.662011 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:45.661985    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:45.662480 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:45.662459    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs" event={"ID":"fdad9cf6-50bd-45e4-87ea-7886d27273f2","Type":"ContainerStarted","Data":"03f7a59e5d0b844473a4443327b59c22a56cb13b4868bba0cbe4424c5718f5be"}
Nov 03 21:55:45.662565 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:45.662526    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"
Nov 03 21:55:45.662565 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:45.662550    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs" event={"ID":"fdad9cf6-50bd-45e4-87ea-7886d27273f2","Type":"ContainerStarted","Data":"8d4fcf8fab211b79db26bc6073bff51116cbc79505b163dcec8a37ba63fbd950"}
Nov 03 21:55:45.663911 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:45.663882    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm" event={"ID":"6254ce84-75c8-40d3-a75b-9df6b08aa18c","Type":"ContainerStarted","Data":"0c8ad895332afc2096a8e99b7cf643fbad8d404d8ed42946f0c4db85533cfd4b"}
Nov 03 21:55:45.664023 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:45.664006    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"
Nov 03 21:55:45.696677 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:45.696629    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb" podStartSLOduration=2.329186772 podStartE2EDuration="2.696616696s" totalImagesPullingTime="367.429924ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:55:45.695804053 +0000 UTC m=+1554.574626342" watchObservedRunningTime="2026-11-03 21:55:45.696616696 +0000 UTC m=+1554.575439002"
Nov 03 21:55:45.738434 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:45.738380    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs" podStartSLOduration=1.738366943 podStartE2EDuration="1.738366943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:55:45.737864838 +0000 UTC m=+1554.616687141" watchObservedRunningTime="2026-11-03 21:55:45.738366943 +0000 UTC m=+1554.617189278"
Nov 03 21:55:45.775586 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:45.775530    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm" podStartSLOduration=2.775517951 podStartE2EDuration="2.775517951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:55:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:55:45.774268196 +0000 UTC m=+1554.653090478" watchObservedRunningTime="2026-11-03 21:55:45.775517951 +0000 UTC m=+1554.654340226"
Nov 03 21:55:47.467422 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:47.467381    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:47.485707 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:47.485063    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:55:49.182933 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:49.182895    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:50.901990 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:50.901945    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:51.674172 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:51.674146    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:55:52.370253 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:55:52.370214    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:55:52.721865 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:52.721768    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:54.549262 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:54.549226    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:56.168178 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:56.168136    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:56.671771 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:56.671734    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"
Nov 03 21:55:56.671970 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:56.671818    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"
Nov 03 21:55:56.892668 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:56.892630    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:57.497013 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:57.496975    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:57.613763 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:57.613731    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:57.713883 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:57.713850    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:57.952924 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:57.952877    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:59.604589 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:59.604554    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:55:59.707929 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:55:59.707895    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:01.587577 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:01.587542    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:02.224906 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:02.224871    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:56:03.370772 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:03.370735    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:05.140171 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:05.140138    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:06.856403 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:06.856367    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:08.567079 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:08.567020    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:10.261388 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:10.261346    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:12.043955 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:12.043914    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:13.657243 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:13.657204    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:15.356856 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:15.356795    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:16.992030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:16.991989    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:18.496463 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:18.496431    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:56:18.764615 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:18.764530    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:20.465469 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:20.465429    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:22.174805 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:22.174765    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:22.370674 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:56:22.370639    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:56:23.861937 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:23.861898    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:25.655347 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:25.655306    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:27.350366 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:27.350330    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:29.065037 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:29.064998    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:30.756019 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:30.755985    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:32.471081 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:32.471047    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:34.157235 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:34.157197    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:35.853029 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:35.852990    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:37.470347 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:37.470312    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:37.923431 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:37.923399    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:39.255788 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:39.255750    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:40.947106 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:40.947071    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:42.205797 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:42.205763    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:42.840962 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:42.840930    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:43.584415 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:43.584335    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:56:44.675406 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:44.675353    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:46.475392 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:46.475351    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:47.240891 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:47.240860    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:48.149678 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:48.149636    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:49.841642 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:49.841602    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:51.467237 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:51.467206    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:52.370600 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:56:52.370565    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:56:52.886330 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:52.886295    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"]
Nov 03 21:56:52.886991 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:52.886535    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb" podUID="0aa9e821-efa5-4e33-a6a1-02aa18ccf284" containerName="maas-api" containerID="cri-o://1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436" gracePeriod=30
Nov 03 21:56:52.934002 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:52.933963    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"]
Nov 03 21:56:52.964726 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:52.964632    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"]
Nov 03 21:56:53.015109 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.015069    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:53.158472 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.158390    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:53.357308 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.357283    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:56:53.460471 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.460381    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-kube-api-access-xt9gn\" (UniqueName: \"kubernetes.io/projected/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-kube-api-access-xt9gn\") pod \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") "
Nov 03 21:56:53.460471 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.460465    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-maas-api-tls\") pod \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") "
Nov 03 21:56:53.460733 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.460488    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-openshift-service-ca\") pod \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") "
Nov 03 21:56:53.460733 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.460539    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-metrics-certs\") pod \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\" (UID: \"0aa9e821-efa5-4e33-a6a1-02aa18ccf284\") "
Nov 03 21:56:53.461258 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.460955    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-openshift-service-ca" pod "0aa9e821-efa5-4e33-a6a1-02aa18ccf284" (UID: "0aa9e821-efa5-4e33-a6a1-02aa18ccf284"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:56:53.462610 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.462587    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-maas-api-tls" pod "0aa9e821-efa5-4e33-a6a1-02aa18ccf284" (UID: "0aa9e821-efa5-4e33-a6a1-02aa18ccf284"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:56:53.462714 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.462677    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-metrics-certs" pod "0aa9e821-efa5-4e33-a6a1-02aa18ccf284" (UID: "0aa9e821-efa5-4e33-a6a1-02aa18ccf284"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:56:53.462817 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.462797    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-kube-api-access-xt9gn" pod "0aa9e821-efa5-4e33-a6a1-02aa18ccf284" (UID: "0aa9e821-efa5-4e33-a6a1-02aa18ccf284"). InnerVolumeSpecName "kube-api-access-xt9gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:56:53.561871 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.561839    4454 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-maas-api-tls\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:56:53.561871 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.561866    4454 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-openshift-service-ca\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:56:53.561871 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.561876    4454 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-metrics-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:56:53.562106 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.561884    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xt9gn\" (UniqueName: \"kubernetes.io/projected/0aa9e821-efa5-4e33-a6a1-02aa18ccf284-kube-api-access-xt9gn\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:56:53.982404 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.982365    4454 generic.go:356] "Generic (PLEG): container finished" podID="0aa9e821-efa5-4e33-a6a1-02aa18ccf284" containerID="1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436" exitCode=0
Nov 03 21:56:53.982901 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.982424    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"
Nov 03 21:56:53.982901 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.982436    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb" event={"ID":"0aa9e821-efa5-4e33-a6a1-02aa18ccf284","Type":"ContainerDied","Data":"1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436"}
Nov 03 21:56:53.982901 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.982497    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb" event={"ID":"0aa9e821-efa5-4e33-a6a1-02aa18ccf284","Type":"ContainerDied","Data":"38e2ebe61baf69d4a7a16a9466dedd39bac3b313cb28fb1701e54b83bdcdaca9"}
Nov 03 21:56:53.982901 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.982531    4454 scope.go:122] "RemoveContainer" containerID="1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436"
Nov 03 21:56:53.991500 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.991481    4454 scope.go:122] "RemoveContainer" containerID="1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436"
Nov 03 21:56:53.991775 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:56:53.991756    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436\": container with ID starting with 1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436 not found: ID does not exist" containerID="1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436"
Nov 03 21:56:53.991814 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:53.991784    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436"} err="failed to get container status \"1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436\": rpc error: code = NotFound desc = could not find container \"1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436\": container with ID starting with 1db5e4537d87f5c359fd88006b595bf4ad3e77008dcc87f841236f661c6ce436 not found: ID does not exist"
Nov 03 21:56:54.028406 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:54.028379    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:54.030174 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:54.030151    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"]
Nov 03 21:56:54.041307 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:54.041281    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9f9b50-645cdcbff9-q5qlb"]
Nov 03 21:56:54.149328 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:54.149293    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:54.669060 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:54.669026    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:56:55.228024 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:55.227994    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0aa9e821-efa5-4e33-a6a1-02aa18ccf284" path="/var/lib/kubelet/pods/0aa9e821-efa5-4e33-a6a1-02aa18ccf284/volumes"
Nov 03 21:56:56.149899 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:56.149859    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:56:57.934701 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:57.934657    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm" podUID="6254ce84-75c8-40d3-a75b-9df6b08aa18c" containerName="payload-processing" containerID="cri-o://0c8ad895332afc2096a8e99b7cf643fbad8d404d8ed42946f0c4db85533cfd4b" gracePeriod=25
Nov 03 21:56:57.965942 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:56:57.965892    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs" podUID="fdad9cf6-50bd-45e4-87ea-7886d27273f2" containerName="payload-pre-processing" containerID="cri-o://03f7a59e5d0b844473a4443327b59c22a56cb13b4868bba0cbe4424c5718f5be" gracePeriod=25
Nov 03 21:57:00.417472 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:00.417434    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:57:00.518979 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:00.518936    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:57:00.743136 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:00.743052    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:57:06.669573 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:06.669533    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm" podUID="6254ce84-75c8-40d3-a75b-9df6b08aa18c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:57:06.669969 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:06.669576    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs" podUID="fdad9cf6-50bd-45e4-87ea-7886d27273f2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:57:14.154250 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:14.154211    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:57:16.669476 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:16.669436    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm" podUID="6254ce84-75c8-40d3-a75b-9df6b08aa18c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:57:16.669876 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:16.669449    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs" podUID="fdad9cf6-50bd-45e4-87ea-7886d27273f2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:57:18.173848 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.172489    4454 generic.go:356] "Generic (PLEG): container finished" podID="6254ce84-75c8-40d3-a75b-9df6b08aa18c" containerID="0c8ad895332afc2096a8e99b7cf643fbad8d404d8ed42946f0c4db85533cfd4b" exitCode=0
Nov 03 21:57:18.175128 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.172648    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm" event={"ID":"6254ce84-75c8-40d3-a75b-9df6b08aa18c","Type":"ContainerDied","Data":"0c8ad895332afc2096a8e99b7cf643fbad8d404d8ed42946f0c4db85533cfd4b"}
Nov 03 21:57:18.176623 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.176535    4454 generic.go:356] "Generic (PLEG): container finished" podID="fdad9cf6-50bd-45e4-87ea-7886d27273f2" containerID="03f7a59e5d0b844473a4443327b59c22a56cb13b4868bba0cbe4424c5718f5be" exitCode=0
Nov 03 21:57:18.176942 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.176870    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs" event={"ID":"fdad9cf6-50bd-45e4-87ea-7886d27273f2","Type":"ContainerDied","Data":"03f7a59e5d0b844473a4443327b59c22a56cb13b4868bba0cbe4424c5718f5be"}
Nov 03 21:57:18.237855 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.237819    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"
Nov 03 21:57:18.278625 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.278591    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"
Nov 03 21:57:18.352184 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.352155    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6254ce84-75c8-40d3-a75b-9df6b08aa18c-kube-api-access-kdjr9\" (UniqueName: \"kubernetes.io/projected/6254ce84-75c8-40d3-a75b-9df6b08aa18c-kube-api-access-kdjr9\") pod \"6254ce84-75c8-40d3-a75b-9df6b08aa18c\" (UID: \"6254ce84-75c8-40d3-a75b-9df6b08aa18c\") "
Nov 03 21:57:18.352362 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.352231    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6254ce84-75c8-40d3-a75b-9df6b08aa18c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6254ce84-75c8-40d3-a75b-9df6b08aa18c-plugins-config-volume\") pod \"6254ce84-75c8-40d3-a75b-9df6b08aa18c\" (UID: \"6254ce84-75c8-40d3-a75b-9df6b08aa18c\") "
Nov 03 21:57:18.352671 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.352642    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6254ce84-75c8-40d3-a75b-9df6b08aa18c-plugins-config-volume" pod "6254ce84-75c8-40d3-a75b-9df6b08aa18c" (UID: "6254ce84-75c8-40d3-a75b-9df6b08aa18c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:57:18.354325 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.354304    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6254ce84-75c8-40d3-a75b-9df6b08aa18c-kube-api-access-kdjr9" pod "6254ce84-75c8-40d3-a75b-9df6b08aa18c" (UID: "6254ce84-75c8-40d3-a75b-9df6b08aa18c"). InnerVolumeSpecName "kube-api-access-kdjr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:57:18.453487 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.453396    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fdad9cf6-50bd-45e4-87ea-7886d27273f2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdad9cf6-50bd-45e4-87ea-7886d27273f2-plugins-config-volume\") pod \"fdad9cf6-50bd-45e4-87ea-7886d27273f2\" (UID: \"fdad9cf6-50bd-45e4-87ea-7886d27273f2\") "
Nov 03 21:57:18.453703 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.453595    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdad9cf6-50bd-45e4-87ea-7886d27273f2-kube-api-access-s68nh\" (UniqueName: \"kubernetes.io/projected/fdad9cf6-50bd-45e4-87ea-7886d27273f2-kube-api-access-s68nh\") pod \"fdad9cf6-50bd-45e4-87ea-7886d27273f2\" (UID: \"fdad9cf6-50bd-45e4-87ea-7886d27273f2\") "
Nov 03 21:57:18.453794 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.453771    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdad9cf6-50bd-45e4-87ea-7886d27273f2-plugins-config-volume" pod "fdad9cf6-50bd-45e4-87ea-7886d27273f2" (UID: "fdad9cf6-50bd-45e4-87ea-7886d27273f2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:57:18.453923 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.453898    4454 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6254ce84-75c8-40d3-a75b-9df6b08aa18c-plugins-config-volume\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:57:18.453923 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.453923    4454 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdad9cf6-50bd-45e4-87ea-7886d27273f2-plugins-config-volume\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:57:18.454084 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.453932    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdjr9\" (UniqueName: \"kubernetes.io/projected/6254ce84-75c8-40d3-a75b-9df6b08aa18c-kube-api-access-kdjr9\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:57:18.455651 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.455627    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdad9cf6-50bd-45e4-87ea-7886d27273f2-kube-api-access-s68nh" pod "fdad9cf6-50bd-45e4-87ea-7886d27273f2" (UID: "fdad9cf6-50bd-45e4-87ea-7886d27273f2"). InnerVolumeSpecName "kube-api-access-s68nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:57:18.554808 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:18.554778    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s68nh\" (UniqueName: \"kubernetes.io/projected/fdad9cf6-50bd-45e4-87ea-7886d27273f2-kube-api-access-s68nh\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 21:57:19.182803 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:19.182767    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"
Nov 03 21:57:19.183284 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:19.182819    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs" event={"ID":"fdad9cf6-50bd-45e4-87ea-7886d27273f2","Type":"ContainerDied","Data":"8d4fcf8fab211b79db26bc6073bff51116cbc79505b163dcec8a37ba63fbd950"}
Nov 03 21:57:19.183284 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:19.182883    4454 scope.go:122] "RemoveContainer" containerID="03f7a59e5d0b844473a4443327b59c22a56cb13b4868bba0cbe4424c5718f5be"
Nov 03 21:57:19.184350 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:19.184325    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm" event={"ID":"6254ce84-75c8-40d3-a75b-9df6b08aa18c","Type":"ContainerDied","Data":"61d0d3cf582dc3783026cc96054e161e517bc541201c1fb3c67a2658e98f3f3f"}
Nov 03 21:57:19.184430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:19.184358    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"
Nov 03 21:57:19.191378 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:19.191359    4454 scope.go:122] "RemoveContainer" containerID="0c8ad895332afc2096a8e99b7cf643fbad8d404d8ed42946f0c4db85533cfd4b"
Nov 03 21:57:19.229376 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:19.229353    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"]
Nov 03 21:57:19.240449 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:19.240424    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-9f9b50-6746dqqfgs"]
Nov 03 21:57:19.264215 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:19.264193    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"]
Nov 03 21:57:19.268990 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:19.268964    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-9f9b50-9c565486bvkzjm"]
Nov 03 21:57:20.081195 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:20.081163    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:57:21.228474 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:21.228443    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6254ce84-75c8-40d3-a75b-9df6b08aa18c" path="/var/lib/kubelet/pods/6254ce84-75c8-40d3-a75b-9df6b08aa18c/volumes"
Nov 03 21:57:21.228867 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:21.228836    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdad9cf6-50bd-45e4-87ea-7886d27273f2" path="/var/lib/kubelet/pods/fdad9cf6-50bd-45e4-87ea-7886d27273f2/volumes"
Nov 03 21:57:22.370717 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:57:22.370678    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:57:23.889683 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:23.889644    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:57:24.096899 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:24.096854    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:57:29.572649 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:29.572607    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:57:38.043638 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:38.043548    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:57:38.151195 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:57:38.151158    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:57:52.370136 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:57:52.370093    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:58:22.370469 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:58:22.370432    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:58:38.172173 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:58:38.172130    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:58:47.187737 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:58:47.187703    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:58:48.710430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:58:48.710336    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:58:48.817493 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:58:48.817455    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:58:52.371327 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:58:52.371277    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:59:02.479979 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:02.479945    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:59:08.291709 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:08.291597    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:59:22.371924 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:59:22.371882    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:59:23.651091 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:23.651059    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:59:24.022978 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:24.022902    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:24.834365 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:24.834324    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:27.974154 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:27.974117    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:28.089319 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:28.089280    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:29.549166 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.549136    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"]
Nov 03 21:59:29.549695 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.549679    4454 cpu_manager.go:414] "containerMap: removing container" podUID="fdad9cf6-50bd-45e4-87ea-7886d27273f2" containerName="payload-pre-processing"
Nov 03 21:59:29.549760 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.549700    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdad9cf6-50bd-45e4-87ea-7886d27273f2" containerName="payload-pre-processing"
Nov 03 21:59:29.549760 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.549731    4454 cpu_manager.go:414] "containerMap: removing container" podUID="6254ce84-75c8-40d3-a75b-9df6b08aa18c" containerName="payload-processing"
Nov 03 21:59:29.549760 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.549738    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6254ce84-75c8-40d3-a75b-9df6b08aa18c" containerName="payload-processing"
Nov 03 21:59:29.549760 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.549751    4454 cpu_manager.go:414] "containerMap: removing container" podUID="0aa9e821-efa5-4e33-a6a1-02aa18ccf284" containerName="maas-api"
Nov 03 21:59:29.549760 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.549757    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0aa9e821-efa5-4e33-a6a1-02aa18ccf284" containerName="maas-api"
Nov 03 21:59:29.549953 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.549861    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdad9cf6-50bd-45e4-87ea-7886d27273f2" containerName="payload-pre-processing"
Nov 03 21:59:29.549953 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.549877    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="6254ce84-75c8-40d3-a75b-9df6b08aa18c" containerName="payload-processing"
Nov 03 21:59:29.549953 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.549885    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="0aa9e821-efa5-4e33-a6a1-02aa18ccf284" containerName="maas-api"
Nov 03 21:59:29.553013 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.552995    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.581114 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.581084    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"]
Nov 03 21:59:29.581966 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.581905    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:29.613460 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.613432    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.613651 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.613485    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-metrics-certs\") pod \"maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.613651 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.613586    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.613742 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.613719    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzw7m\" (UniqueName: \"kubernetes.io/projected/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-kube-api-access-jzw7m\") pod \"maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.641339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.641307    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-3503c0-serving-cert\""
Nov 03 21:59:29.678357 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.678322    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"]
Nov 03 21:59:29.681914 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.681892    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"
Nov 03 21:59:29.697436 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.697397    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:29.701443 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.701422    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-3503c0\""
Nov 03 21:59:29.701910 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.701893    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-a9909daf-dockercfg-zkd65\""
Nov 03 21:59:29.703175 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.703156    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"]
Nov 03 21:59:29.714202 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.714172    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzw7m\" (UniqueName: \"kubernetes.io/projected/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-kube-api-access-jzw7m\") pod \"maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.714302 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.714220    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9v2c\" (UniqueName: \"kubernetes.io/projected/32c1cf85-0e06-4194-aa65-800e01880ad6-kube-api-access-w9v2c\") pod \"payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf\" (UID: \"32c1cf85-0e06-4194-aa65-800e01880ad6\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"
Nov 03 21:59:29.714302 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.714275    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.714380 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.714302    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32c1cf85-0e06-4194-aa65-800e01880ad6-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf\" (UID: \"32c1cf85-0e06-4194-aa65-800e01880ad6\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"
Nov 03 21:59:29.714465 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.714447    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-metrics-certs\") pod \"maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.714536 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.714477    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.715246 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.715219    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.716792 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.716771    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-metrics-certs\") pod \"maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.716895 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.716877    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.732859 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.732836    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzw7m\" (UniqueName: \"kubernetes.io/projected/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-kube-api-access-jzw7m\") pod \"maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.734190 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.734138    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"]
Nov 03 21:59:29.738008 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.737984    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"
Nov 03 21:59:29.760688 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.760660    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"]
Nov 03 21:59:29.798593 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.798553    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:29.816555 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.816320    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6608b09e-e647-45c0-b0a3-536c870aa813-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc\" (UID: \"6608b09e-e647-45c0-b0a3-536c870aa813\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"
Nov 03 21:59:29.816721 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.816568    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9v2c\" (UniqueName: \"kubernetes.io/projected/32c1cf85-0e06-4194-aa65-800e01880ad6-kube-api-access-w9v2c\") pod \"payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf\" (UID: \"32c1cf85-0e06-4194-aa65-800e01880ad6\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"
Nov 03 21:59:29.816721 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.816612    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hfnh\" (UniqueName: \"kubernetes.io/projected/6608b09e-e647-45c0-b0a3-536c870aa813-kube-api-access-2hfnh\") pod \"payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc\" (UID: \"6608b09e-e647-45c0-b0a3-536c870aa813\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"
Nov 03 21:59:29.816721 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.816659    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32c1cf85-0e06-4194-aa65-800e01880ad6-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf\" (UID: \"32c1cf85-0e06-4194-aa65-800e01880ad6\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"
Nov 03 21:59:29.817578 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.817551    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32c1cf85-0e06-4194-aa65-800e01880ad6-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf\" (UID: \"32c1cf85-0e06-4194-aa65-800e01880ad6\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"
Nov 03 21:59:29.836156 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.836121    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9v2c\" (UniqueName: \"kubernetes.io/projected/32c1cf85-0e06-4194-aa65-800e01880ad6-kube-api-access-w9v2c\") pod \"payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf\" (UID: \"32c1cf85-0e06-4194-aa65-800e01880ad6\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"
Nov 03 21:59:29.868236 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.868205    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:29.918126 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.918091    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6608b09e-e647-45c0-b0a3-536c870aa813-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc\" (UID: \"6608b09e-e647-45c0-b0a3-536c870aa813\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"
Nov 03 21:59:29.918230 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.918160    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hfnh\" (UniqueName: \"kubernetes.io/projected/6608b09e-e647-45c0-b0a3-536c870aa813-kube-api-access-2hfnh\") pod \"payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc\" (UID: \"6608b09e-e647-45c0-b0a3-536c870aa813\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"
Nov 03 21:59:29.919469 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.919438    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6608b09e-e647-45c0-b0a3-536c870aa813-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc\" (UID: \"6608b09e-e647-45c0-b0a3-536c870aa813\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"
Nov 03 21:59:29.936903 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.936847    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hfnh\" (UniqueName: \"kubernetes.io/projected/6608b09e-e647-45c0-b0a3-536c870aa813-kube-api-access-2hfnh\") pod \"payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc\" (UID: \"6608b09e-e647-45c0-b0a3-536c870aa813\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"
Nov 03 21:59:29.994521 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:29.994472    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"
Nov 03 21:59:30.052843 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.052560    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"
Nov 03 21:59:30.170388 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.170352    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"]
Nov 03 21:59:30.175458 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:59:30.175428    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c640a3_e23a_4a5f_bfc6_4b5068c7278e.slice/crio-eeaaba3035eeeec774fb21f73ef11e81258b99c33799015b018e0bc145cb0749 WatchSource:0}: Error finding container eeaaba3035eeeec774fb21f73ef11e81258b99c33799015b018e0bc145cb0749: Status 404 returned error can't find the container with id eeaaba3035eeeec774fb21f73ef11e81258b99c33799015b018e0bc145cb0749
Nov 03 21:59:30.286083 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.286047    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"]
Nov 03 21:59:30.290418 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:59:30.290390    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c1cf85_0e06_4194_aa65_800e01880ad6.slice/crio-48afbd9a41dd9bc5963a642c1ed215bd777af034c2e259586b99f4f76720f78d WatchSource:0}: Error finding container 48afbd9a41dd9bc5963a642c1ed215bd777af034c2e259586b99f4f76720f78d: Status 404 returned error can't find the container with id 48afbd9a41dd9bc5963a642c1ed215bd777af034c2e259586b99f4f76720f78d
Nov 03 21:59:30.347215 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.347130    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"]
Nov 03 21:59:30.353164 ip-10-0-137-212 kubenswrapper[4454]: W1103 21:59:30.353086    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6608b09e_e647_45c0_b0a3_536c870aa813.slice/crio-471ca206541eb9e07c20818e8c7450409bd209da454acb618e3d64ba51439c3d WatchSource:0}: Error finding container 471ca206541eb9e07c20818e8c7450409bd209da454acb618e3d64ba51439c3d: Status 404 returned error can't find the container with id 471ca206541eb9e07c20818e8c7450409bd209da454acb618e3d64ba51439c3d
Nov 03 21:59:30.829659 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.829624    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf" event={"ID":"32c1cf85-0e06-4194-aa65-800e01880ad6","Type":"ContainerStarted","Data":"8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d"}
Nov 03 21:59:30.830110 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.829674    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf" event={"ID":"32c1cf85-0e06-4194-aa65-800e01880ad6","Type":"ContainerStarted","Data":"48afbd9a41dd9bc5963a642c1ed215bd777af034c2e259586b99f4f76720f78d"}
Nov 03 21:59:30.830110 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.829720    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"
Nov 03 21:59:30.831179 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.831154    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv" event={"ID":"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e","Type":"ContainerStarted","Data":"80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319"}
Nov 03 21:59:30.831312 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.831204    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:30.831312 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.831226    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv" event={"ID":"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e","Type":"ContainerStarted","Data":"eeaaba3035eeeec774fb21f73ef11e81258b99c33799015b018e0bc145cb0749"}
Nov 03 21:59:30.832702 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.832680    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc" event={"ID":"6608b09e-e647-45c0-b0a3-536c870aa813","Type":"ContainerStarted","Data":"8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22"}
Nov 03 21:59:30.832807 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.832714    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc" event={"ID":"6608b09e-e647-45c0-b0a3-536c870aa813","Type":"ContainerStarted","Data":"471ca206541eb9e07c20818e8c7450409bd209da454acb618e3d64ba51439c3d"}
Nov 03 21:59:30.832807 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.832735    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"
Nov 03 21:59:30.866376 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.866334    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf" podStartSLOduration=1.8663210399999999 podStartE2EDuration="1.86632104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:30.865563002 +0000 UTC m=+1779.744385288" watchObservedRunningTime="2026-11-03 21:59:30.86632104 +0000 UTC m=+1779.745143325"
Nov 03 21:59:30.902658 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:30.902558    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc" podStartSLOduration=1.90254644 podStartE2EDuration="1.90254644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:30.901146996 +0000 UTC m=+1779.779969284" watchObservedRunningTime="2026-11-03 21:59:30.90254644 +0000 UTC m=+1779.781368724"
Nov 03 21:59:31.556218 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:31.556175    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:33.274547 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:33.274468    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:33.397067 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:33.397008    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv" podStartSLOduration=3.935553325 podStartE2EDuration="4.396993771s" totalImagesPullingTime="461.440446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:30.937501454 +0000 UTC m=+1779.816323739" watchObservedRunningTime="2026-11-03 21:59:33.396993771 +0000 UTC m=+1782.275816056"
Nov 03 21:59:33.399071 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:33.399036    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:59:35.016430 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:35.016382    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:36.782366 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:36.782330    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:36.844159 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:36.844130    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 21:59:38.305223 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:38.305182    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:38.611485 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:38.611452    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:40.553633 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:40.553592    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:41.839976 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:41.839948    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"
Nov 03 21:59:41.840468 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:41.840038    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"
Nov 03 21:59:42.275339 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:42.275296    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:42.782882 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:42.782842    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:44.023850 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:44.023814    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:45.782584 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:45.782544    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:47.595599 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:47.595562    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:49.389153 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:49.389114    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:51.155151 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:51.155113    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:51.389501 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:51.389468    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:59:51.394607 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:51.394585    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 21:59:52.370971 ip-10-0-137-212 kubenswrapper[4454]: E1103 21:59:52.370936    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 21:59:52.764682 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:52.764601    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:53.349357 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:53.349314    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:53.457463 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:53.457428    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:53.558602 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:53.558561    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:54.524030 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:54.523992    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:54.881913 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:54.881871    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:56.250479 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:56.250443    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:57.067103 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:57.067060    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 21:59:58.056668 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:58.056632    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 21:59:59.752729 ip-10-0-137-212 kubenswrapper[4454]: I1103 21:59:59.752692    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:01.475008 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:01.474963    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:03.260395 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:03.260356    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:05.005166 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:05.005131    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:06.649569 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:06.649532    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:08.363124 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:08.363082    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:10.052203 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:10.052160    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:11.760016 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:11.759978    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:13.188150 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:13.188114    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:00:13.646126 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:13.646085    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:15.415374 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:15.415318    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:16.973875 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:16.973837    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:18.753470 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:18.753432    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:20.452524 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:20.452473    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:22.144048 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:22.144008    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:22.371981 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:00:22.371945    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:00:23.951683 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:23.951650    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:25.761867 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:25.761827    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:27.561441 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:27.561397    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:29.240613 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:29.240575    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:30.937710 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:30.937667    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:32.656117 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:32.656078    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:34.265018 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:34.264934    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:36.028309 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:36.028270    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:36.142851 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:36.142800    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:36.707776 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:36.707743    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:00:37.932525 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:37.932465    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:38.432492 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:38.432456    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:39.760598 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:39.760559    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:40.992306 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:40.992268    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:41.467897 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:41.467852    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:43.145894 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:43.145848    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:44.850393 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:44.850359    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:46.556154 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:46.556110    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:46.582556 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:46.582526    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"]
Nov 03 22:00:46.609496 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:46.609464    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"]
Nov 03 22:00:46.710152 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:46.710106    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:46.819370 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:46.819281    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:48.255252 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:48.255210    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:49.676353 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:49.676316    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"]
Nov 03 22:00:49.777388 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:49.777350    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:50.210864 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.210799    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv" podUID="f1c640a3-e23a-4a5f-bfc6-4b5068c7278e" containerName="maas-api" containerID="cri-o://80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319" gracePeriod=30
Nov 03 22:00:50.243194 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.243158    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:50.343481 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.343439    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:50.464001 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.463941    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 22:00:50.542982 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.542952    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-openshift-service-ca\") pod \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") "
Nov 03 22:00:50.542982 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.542989    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-metrics-certs\") pod \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") "
Nov 03 22:00:50.543200 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.543175    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-kube-api-access-jzw7m\" (UniqueName: \"kubernetes.io/projected/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-kube-api-access-jzw7m\") pod \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") "
Nov 03 22:00:50.543269 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.543252    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-maas-api-tls\") pod \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\" (UID: \"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e\") "
Nov 03 22:00:50.543363 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.543334    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-openshift-service-ca" pod "f1c640a3-e23a-4a5f-bfc6-4b5068c7278e" (UID: "f1c640a3-e23a-4a5f-bfc6-4b5068c7278e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:00:50.543611 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.543592    4454 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-openshift-service-ca\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:00:50.545039 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.545018    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-metrics-certs" pod "f1c640a3-e23a-4a5f-bfc6-4b5068c7278e" (UID: "f1c640a3-e23a-4a5f-bfc6-4b5068c7278e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:00:50.545115 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.545101    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-kube-api-access-jzw7m" pod "f1c640a3-e23a-4a5f-bfc6-4b5068c7278e" (UID: "f1c640a3-e23a-4a5f-bfc6-4b5068c7278e"). InnerVolumeSpecName "kube-api-access-jzw7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:00:50.545211 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.545194    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-maas-api-tls" pod "f1c640a3-e23a-4a5f-bfc6-4b5068c7278e" (UID: "f1c640a3-e23a-4a5f-bfc6-4b5068c7278e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:00:50.644529 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.644475    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzw7m\" (UniqueName: \"kubernetes.io/projected/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-kube-api-access-jzw7m\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:00:50.644529 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.644502    4454 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-maas-api-tls\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:00:50.644529 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.644538    4454 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e-metrics-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:00:50.899638 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:50.899607    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:51.216700 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.216610    4454 generic.go:356] "Generic (PLEG): container finished" podID="f1c640a3-e23a-4a5f-bfc6-4b5068c7278e" containerID="80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319" exitCode=0
Nov 03 22:00:51.216700 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.216659    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"
Nov 03 22:00:51.216700 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.216677    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv" event={"ID":"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e","Type":"ContainerDied","Data":"80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319"}
Nov 03 22:00:51.216973 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.216726    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv" event={"ID":"f1c640a3-e23a-4a5f-bfc6-4b5068c7278e","Type":"ContainerDied","Data":"eeaaba3035eeeec774fb21f73ef11e81258b99c33799015b018e0bc145cb0749"}
Nov 03 22:00:51.216973 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.216750    4454 scope.go:122] "RemoveContainer" containerID="80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319"
Nov 03 22:00:51.226331 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.226305    4454 scope.go:122] "RemoveContainer" containerID="80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319"
Nov 03 22:00:51.226694 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:00:51.226672    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319\": container with ID starting with 80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319 not found: ID does not exist" containerID="80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319"
Nov 03 22:00:51.226788 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.226701    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319"} err="failed to get container status \"80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319\": rpc error: code = NotFound desc = could not find container \"80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319\": container with ID starting with 80991fe4f47fa52f8b17a4263d73197a3b48019bf1585f7b1937ba37cb951319 not found: ID does not exist"
Nov 03 22:00:51.263935 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.263884    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"]
Nov 03 22:00:51.274978 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.274950    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3503c0-55c4fb769b-9c4fv"]
Nov 03 22:00:51.583919 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.583832    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf" podUID="32c1cf85-0e06-4194-aa65-800e01880ad6" containerName="payload-processing" containerID="cri-o://8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d" gracePeriod=25
Nov 03 22:00:51.609773 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.609740    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc" podUID="6608b09e-e647-45c0-b0a3-536c870aa813" containerName="payload-pre-processing" containerID="cri-o://8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22" gracePeriod=25
Nov 03 22:00:51.851491 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.851466    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"
Nov 03 22:00:51.881971 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.881935    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"
Nov 03 22:00:51.958567 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.958534    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32c1cf85-0e06-4194-aa65-800e01880ad6-kube-api-access-w9v2c\" (UniqueName: \"kubernetes.io/projected/32c1cf85-0e06-4194-aa65-800e01880ad6-kube-api-access-w9v2c\") pod \"32c1cf85-0e06-4194-aa65-800e01880ad6\" (UID: \"32c1cf85-0e06-4194-aa65-800e01880ad6\") "
Nov 03 22:00:51.958954 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.958612    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6608b09e-e647-45c0-b0a3-536c870aa813-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6608b09e-e647-45c0-b0a3-536c870aa813-plugins-config-volume\") pod \"6608b09e-e647-45c0-b0a3-536c870aa813\" (UID: \"6608b09e-e647-45c0-b0a3-536c870aa813\") "
Nov 03 22:00:51.958954 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.958667    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32c1cf85-0e06-4194-aa65-800e01880ad6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32c1cf85-0e06-4194-aa65-800e01880ad6-plugins-config-volume\") pod \"32c1cf85-0e06-4194-aa65-800e01880ad6\" (UID: \"32c1cf85-0e06-4194-aa65-800e01880ad6\") "
Nov 03 22:00:51.958954 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.958691    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6608b09e-e647-45c0-b0a3-536c870aa813-kube-api-access-2hfnh\" (UniqueName: \"kubernetes.io/projected/6608b09e-e647-45c0-b0a3-536c870aa813-kube-api-access-2hfnh\") pod \"6608b09e-e647-45c0-b0a3-536c870aa813\" (UID: \"6608b09e-e647-45c0-b0a3-536c870aa813\") "
Nov 03 22:00:51.959067 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.959032    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6608b09e-e647-45c0-b0a3-536c870aa813-plugins-config-volume" pod "6608b09e-e647-45c0-b0a3-536c870aa813" (UID: "6608b09e-e647-45c0-b0a3-536c870aa813"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:00:51.959112 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.959063    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32c1cf85-0e06-4194-aa65-800e01880ad6-plugins-config-volume" pod "32c1cf85-0e06-4194-aa65-800e01880ad6" (UID: "32c1cf85-0e06-4194-aa65-800e01880ad6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:00:51.960580 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.960558    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c1cf85-0e06-4194-aa65-800e01880ad6-kube-api-access-w9v2c" pod "32c1cf85-0e06-4194-aa65-800e01880ad6" (UID: "32c1cf85-0e06-4194-aa65-800e01880ad6"). InnerVolumeSpecName "kube-api-access-w9v2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:00:51.960651 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:51.960631    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6608b09e-e647-45c0-b0a3-536c870aa813-kube-api-access-2hfnh" pod "6608b09e-e647-45c0-b0a3-536c870aa813" (UID: "6608b09e-e647-45c0-b0a3-536c870aa813"). InnerVolumeSpecName "kube-api-access-2hfnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:00:52.059548 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.059496    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hfnh\" (UniqueName: \"kubernetes.io/projected/6608b09e-e647-45c0-b0a3-536c870aa813-kube-api-access-2hfnh\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:00:52.059548 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.059549    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9v2c\" (UniqueName: \"kubernetes.io/projected/32c1cf85-0e06-4194-aa65-800e01880ad6-kube-api-access-w9v2c\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:00:52.059748 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.059559    4454 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6608b09e-e647-45c0-b0a3-536c870aa813-plugins-config-volume\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:00:52.059748 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.059568    4454 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32c1cf85-0e06-4194-aa65-800e01880ad6-plugins-config-volume\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:00:52.101826 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.101598    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:00:52.221302 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.221267    4454 generic.go:356] "Generic (PLEG): container finished" podID="32c1cf85-0e06-4194-aa65-800e01880ad6" containerID="8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d" exitCode=0
Nov 03 22:00:52.221487 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.221314    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"
Nov 03 22:00:52.221487 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.221363    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf" event={"ID":"32c1cf85-0e06-4194-aa65-800e01880ad6","Type":"ContainerDied","Data":"8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d"}
Nov 03 22:00:52.221487 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.221428    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf" event={"ID":"32c1cf85-0e06-4194-aa65-800e01880ad6","Type":"ContainerDied","Data":"48afbd9a41dd9bc5963a642c1ed215bd777af034c2e259586b99f4f76720f78d"}
Nov 03 22:00:52.221487 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.221467    4454 scope.go:122] "RemoveContainer" containerID="8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d"
Nov 03 22:00:52.224121 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.224094    4454 generic.go:356] "Generic (PLEG): container finished" podID="6608b09e-e647-45c0-b0a3-536c870aa813" containerID="8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22" exitCode=0
Nov 03 22:00:52.224230 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.224143    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc" event={"ID":"6608b09e-e647-45c0-b0a3-536c870aa813","Type":"ContainerDied","Data":"8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22"}
Nov 03 22:00:52.224230 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.224159    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"
Nov 03 22:00:52.224230 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.224187    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc" event={"ID":"6608b09e-e647-45c0-b0a3-536c870aa813","Type":"ContainerDied","Data":"471ca206541eb9e07c20818e8c7450409bd209da454acb618e3d64ba51439c3d"}
Nov 03 22:00:52.231312 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.231295    4454 scope.go:122] "RemoveContainer" containerID="8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d"
Nov 03 22:00:52.231642 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:00:52.231620    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d\": container with ID starting with 8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d not found: ID does not exist" containerID="8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d"
Nov 03 22:00:52.231727 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.231646    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d"} err="failed to get container status \"8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d\": rpc error: code = NotFound desc = could not find container \"8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d\": container with ID starting with 8f01da7d1c91b3e3776f08fd3827cc2888058b59a113fdb3e6f0fdf4d673d35d not found: ID does not exist"
Nov 03 22:00:52.231727 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.231664    4454 scope.go:122] "RemoveContainer" containerID="8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22"
Nov 03 22:00:52.239491 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.239472    4454 scope.go:122] "RemoveContainer" containerID="8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22"
Nov 03 22:00:52.239742 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:00:52.239724    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22\": container with ID starting with 8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22 not found: ID does not exist" containerID="8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22"
Nov 03 22:00:52.239797 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.239748    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22"} err="failed to get container status \"8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22\": rpc error: code = NotFound desc = could not find container \"8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22\": container with ID starting with 8e254a95b0a26db34cd99199cd1697679b7f63f5b679e30815d7941e7518fa22 not found: ID does not exist"
Nov 03 22:00:52.267879 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.267857    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"]
Nov 03 22:00:52.269972 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.269936    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc"]
Nov 03 22:00:52.312636 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.312592    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"]
Nov 03 22:00:52.313897 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.313868    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf"]
Nov 03 22:00:52.371723 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:00:52.371638    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:00:52.837829 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.837710    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3503c0-56bd7988f7hckf" podUID="32c1cf85-0e06-4194-aa65-800e01880ad6" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.81:50052\" within 1s: context deadline exceeded"
Nov 03 22:00:52.837829 ip-10-0-137-212 kubenswrapper[4454]: W1103 22:00:52.837790    4454 logging.go:55] [core] [Channel #1061 SubChannel #1064]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.82:50052", ServerName: "10.133.0.82:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.82:50052: operation was canceled"
Nov 03 22:00:52.837829 ip-10-0-137-212 kubenswrapper[4454]: W1103 22:00:52.837790    4454 logging.go:55] [core] [Channel #1062 SubChannel #1063]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.81:50052", ServerName: "10.133.0.81:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.81:50052: operation was canceled"
Nov 03 22:00:52.837829 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:52.837710    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3503c0-8485c4nkzc" podUID="6608b09e-e647-45c0-b0a3-536c870aa813" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.82:50052\" within 1s: context deadline exceeded"
Nov 03 22:00:53.228673 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:53.228639    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32c1cf85-0e06-4194-aa65-800e01880ad6" path="/var/lib/kubelet/pods/32c1cf85-0e06-4194-aa65-800e01880ad6/volumes"
Nov 03 22:00:53.229061 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:53.229009    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6608b09e-e647-45c0-b0a3-536c870aa813" path="/var/lib/kubelet/pods/6608b09e-e647-45c0-b0a3-536c870aa813/volumes"
Nov 03 22:00:53.229424 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:53.229406    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1c640a3-e23a-4a5f-bfc6-4b5068c7278e" path="/var/lib/kubelet/pods/f1c640a3-e23a-4a5f-bfc6-4b5068c7278e/volumes"
Nov 03 22:00:56.551041 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:56.551004    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:56.659819 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:56.659779    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:00:56.876013 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:00:56.875981    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:01:02.421293 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.421257    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"]
Nov 03 22:01:02.421858 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.421739    4454 cpu_manager.go:414] "containerMap: removing container" podUID="f1c640a3-e23a-4a5f-bfc6-4b5068c7278e" containerName="maas-api"
Nov 03 22:01:02.421858 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.421755    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1c640a3-e23a-4a5f-bfc6-4b5068c7278e" containerName="maas-api"
Nov 03 22:01:02.421858 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.421771    4454 cpu_manager.go:414] "containerMap: removing container" podUID="32c1cf85-0e06-4194-aa65-800e01880ad6" containerName="payload-processing"
Nov 03 22:01:02.421858 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.421776    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32c1cf85-0e06-4194-aa65-800e01880ad6" containerName="payload-processing"
Nov 03 22:01:02.421858 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.421784    4454 cpu_manager.go:414] "containerMap: removing container" podUID="6608b09e-e647-45c0-b0a3-536c870aa813" containerName="payload-pre-processing"
Nov 03 22:01:02.421858 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.421792    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6608b09e-e647-45c0-b0a3-536c870aa813" containerName="payload-pre-processing"
Nov 03 22:01:02.422232 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.421908    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1c640a3-e23a-4a5f-bfc6-4b5068c7278e" containerName="maas-api"
Nov 03 22:01:02.422232 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.421921    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="32c1cf85-0e06-4194-aa65-800e01880ad6" containerName="payload-processing"
Nov 03 22:01:02.422232 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.421928    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="6608b09e-e647-45c0-b0a3-536c870aa813" containerName="payload-pre-processing"
Nov 03 22:01:02.426530 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.426501    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.443798 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.443767    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"]
Nov 03 22:01:02.453790 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.453762    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-5efb7ae3-kserve-self-signed-certs\""
Nov 03 22:01:02.474965 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.474926    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:01:02.568345 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.568311    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-dshm\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.568345 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.568351    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjtg7\" (UniqueName: \"kubernetes.io/projected/c8973a6f-ec14-46f1-ba14-54b629936cfd-kube-api-access-jjtg7\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.568571 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.568434    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-home\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.568571 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.568534    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-model-cache\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.568644 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.568629    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-tmp-dir\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.568738 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.568721    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8973a6f-ec14-46f1-ba14-54b629936cfd-tls-certs\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.669797 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.669761    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8973a6f-ec14-46f1-ba14-54b629936cfd-tls-certs\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.670005 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.669806    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-dshm\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.670005 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.669928    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjtg7\" (UniqueName: \"kubernetes.io/projected/c8973a6f-ec14-46f1-ba14-54b629936cfd-kube-api-access-jjtg7\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.670134 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.670004    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-home\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.670134 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.670039    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-model-cache\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.670134 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.670124    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-tmp-dir\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.670443 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.670415    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-home\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.670443 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.670437    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-model-cache\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.670615 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.670473    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-tmp-dir\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.672069 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.672020    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-dshm\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.672325 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.672304    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8973a6f-ec14-46f1-ba14-54b629936cfd-tls-certs\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.687460 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.687430    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjtg7\" (UniqueName: \"kubernetes.io/projected/c8973a6f-ec14-46f1-ba14-54b629936cfd-kube-api-access-jjtg7\") pod \"e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") " pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:02.739044 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:02.739010    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:03.031867 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:03.031777    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"]
Nov 03 22:01:03.036937 ip-10-0-137-212 kubenswrapper[4454]: W1103 22:01:03.036909    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8973a6f_ec14_46f1_ba14_54b629936cfd.slice/crio-e731723f0cf1956c990c2917d8499da704312bc756e9bad02d0945bfc209f921 WatchSource:0}: Error finding container e731723f0cf1956c990c2917d8499da704312bc756e9bad02d0945bfc209f921: Status 404 returned error can't find the container with id e731723f0cf1956c990c2917d8499da704312bc756e9bad02d0945bfc209f921
Nov 03 22:01:03.282206 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:03.282124    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt" event={"ID":"c8973a6f-ec14-46f1-ba14-54b629936cfd","Type":"ContainerStarted","Data":"352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501"}
Nov 03 22:01:03.282206 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:03.282164    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt" event={"ID":"c8973a6f-ec14-46f1-ba14-54b629936cfd","Type":"ContainerStarted","Data":"e731723f0cf1956c990c2917d8499da704312bc756e9bad02d0945bfc209f921"}
Nov 03 22:01:03.319118 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:03.319062    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt" podStartSLOduration=1.31904638 podStartE2EDuration="1.31904638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:01:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:01:03.317679014 +0000 UTC m=+1872.196501299" watchObservedRunningTime="2026-11-03 22:01:03.31904638 +0000 UTC m=+1872.197868664"
Nov 03 22:01:03.739701 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:03.739665    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:04.677470 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:04.677432    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:01:11.648119 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:11.648078    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:01:12.739851 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:12.739804    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:12.752318 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:12.752292    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:13.359917 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:13.359887    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:01:14.527698 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:14.527661    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:01:17.350553 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:17.350490    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:01:17.893752 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:17.893716    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:01:17.995401 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:17.995364    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:01:20.264041 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:20.263999    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:01:22.442122 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:01:22.442083    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:01:23.628151 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:23.628114    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:01:36.058499 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:36.058456    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:01:38.693533 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:38.691190    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:01:39.196542 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:39.195615    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lq9lw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:01:39.196542 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:39.195700    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lq9lw" podUID="6174dd8c-e88b-48eb-8ba6-9935eb711810" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:01:39.217639 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:39.217598    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x4f2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:01:39.217925 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:39.217894    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x4f2g" podUID="ea097b76-14e5-48e9-9966-ed40754d66f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:01:47.912567 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:47.912523    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"]
Nov 03 22:01:47.955920 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:47.955888    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:01:49.993470 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:49.993428    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:01:52.441477 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:01:52.441431    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:01:53.838060 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:53.838014    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:01:59.597445 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:01:59.597408    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:02:02.933968 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:02.933910    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt" podUID="c8973a6f-ec14-46f1-ba14-54b629936cfd" containerName="main" containerID="cri-o://352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501" gracePeriod=45
Nov 03 22:02:03.175575 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.175551    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:02:03.271414 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.271327    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-home\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-home\") pod \"c8973a6f-ec14-46f1-ba14-54b629936cfd\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") "
Nov 03 22:02:03.271414 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.271370    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8973a6f-ec14-46f1-ba14-54b629936cfd-kube-api-access-jjtg7\" (UniqueName: \"kubernetes.io/projected/c8973a6f-ec14-46f1-ba14-54b629936cfd-kube-api-access-jjtg7\") pod \"c8973a6f-ec14-46f1-ba14-54b629936cfd\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") "
Nov 03 22:02:03.271414 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.271390    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8973a6f-ec14-46f1-ba14-54b629936cfd-tls-certs\" (UniqueName: \"kubernetes.io/secret/c8973a6f-ec14-46f1-ba14-54b629936cfd-tls-certs\") pod \"c8973a6f-ec14-46f1-ba14-54b629936cfd\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") "
Nov 03 22:02:03.271729 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.271485    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-model-cache\") pod \"c8973a6f-ec14-46f1-ba14-54b629936cfd\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") "
Nov 03 22:02:03.271729 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.271531    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-dshm\") pod \"c8973a6f-ec14-46f1-ba14-54b629936cfd\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") "
Nov 03 22:02:03.271729 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.271559    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-tmp-dir\") pod \"c8973a6f-ec14-46f1-ba14-54b629936cfd\" (UID: \"c8973a6f-ec14-46f1-ba14-54b629936cfd\") "
Nov 03 22:02:03.271729 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.271654    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-home" pod "c8973a6f-ec14-46f1-ba14-54b629936cfd" (UID: "c8973a6f-ec14-46f1-ba14-54b629936cfd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:02:03.271944 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.271816    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-model-cache" pod "c8973a6f-ec14-46f1-ba14-54b629936cfd" (UID: "c8973a6f-ec14-46f1-ba14-54b629936cfd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:02:03.271944 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.271842    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-tmp-dir" pod "c8973a6f-ec14-46f1-ba14-54b629936cfd" (UID: "c8973a6f-ec14-46f1-ba14-54b629936cfd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:02:03.271944 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.271885    4454 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-home\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:02:03.271944 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.271898    4454 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-model-cache\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:02:03.273741 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.273720    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-dshm" pod "c8973a6f-ec14-46f1-ba14-54b629936cfd" (UID: "c8973a6f-ec14-46f1-ba14-54b629936cfd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:02:03.273841 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.273740    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8973a6f-ec14-46f1-ba14-54b629936cfd-kube-api-access-jjtg7" pod "c8973a6f-ec14-46f1-ba14-54b629936cfd" (UID: "c8973a6f-ec14-46f1-ba14-54b629936cfd"). InnerVolumeSpecName "kube-api-access-jjtg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:02:03.273841 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.273795    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8973a6f-ec14-46f1-ba14-54b629936cfd-tls-certs" pod "c8973a6f-ec14-46f1-ba14-54b629936cfd" (UID: "c8973a6f-ec14-46f1-ba14-54b629936cfd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:02:03.372384 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.372354    4454 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-dshm\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:02:03.372384 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.372379    4454 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8973a6f-ec14-46f1-ba14-54b629936cfd-tmp-dir\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:02:03.372384 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.372389    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjtg7\" (UniqueName: \"kubernetes.io/projected/c8973a6f-ec14-46f1-ba14-54b629936cfd-kube-api-access-jjtg7\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:02:03.372636 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.372399    4454 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8973a6f-ec14-46f1-ba14-54b629936cfd-tls-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:02:03.663552 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.663500    4454 generic.go:356] "Generic (PLEG): container finished" podID="c8973a6f-ec14-46f1-ba14-54b629936cfd" containerID="352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501" exitCode=0
Nov 03 22:02:03.663552 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.663539    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt" event={"ID":"c8973a6f-ec14-46f1-ba14-54b629936cfd","Type":"ContainerDied","Data":"352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501"}
Nov 03 22:02:03.663802 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.663562    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"
Nov 03 22:02:03.663802 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.663594    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt" event={"ID":"c8973a6f-ec14-46f1-ba14-54b629936cfd","Type":"ContainerDied","Data":"e731723f0cf1956c990c2917d8499da704312bc756e9bad02d0945bfc209f921"}
Nov 03 22:02:03.663802 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.663616    4454 scope.go:122] "RemoveContainer" containerID="352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501"
Nov 03 22:02:03.672292 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.672275    4454 scope.go:122] "RemoveContainer" containerID="352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501"
Nov 03 22:02:03.672590 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:02:03.672567    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501\": container with ID starting with 352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501 not found: ID does not exist" containerID="352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501"
Nov 03 22:02:03.672653 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.672603    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501"} err="failed to get container status \"352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501\": rpc error: code = NotFound desc = could not find container \"352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501\": container with ID starting with 352f3d9c10176eac166190194d040086667098b541e1238435221794b6fac501 not found: ID does not exist"
Nov 03 22:02:03.711069 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.711038    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"]
Nov 03 22:02:03.721375 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:03.721351    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-5efb7ae3-kserve-6bdcdcf8f-bctzt"]
Nov 03 22:02:05.228648 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:05.228620    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8973a6f-ec14-46f1-ba14-54b629936cfd" path="/var/lib/kubelet/pods/c8973a6f-ec14-46f1-ba14-54b629936cfd/volumes"
Nov 03 22:02:10.153086 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:10.153048    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:02:14.880452 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:14.880413    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:02:22.370968 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:02:22.370930    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:02:23.672799 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:23.672762    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:02:24.999532 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:24.994198    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:02:25.086067 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:25.086028    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:02:26.082531 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:26.081634    4454 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-x7pth container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:02:26.082531 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:26.081769    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-x7pth" podUID="e86fb6a8-2f81-4752-92bb-adff8cd0f2cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:02:38.805134 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:38.805092    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:02:40.481167 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:40.481132    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:02:41.654632 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.654595    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"]
Nov 03 22:02:41.655136 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.655112    4454 cpu_manager.go:414] "containerMap: removing container" podUID="c8973a6f-ec14-46f1-ba14-54b629936cfd" containerName="main"
Nov 03 22:02:41.655211 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.655136    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8973a6f-ec14-46f1-ba14-54b629936cfd" containerName="main"
Nov 03 22:02:41.655264 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.655222    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8973a6f-ec14-46f1-ba14-54b629936cfd" containerName="main"
Nov 03 22:02:41.658365 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.658345    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.685223 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.685190    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"]
Nov 03 22:02:41.779887 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.779849    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzh9k\" (UniqueName: \"kubernetes.io/projected/b5927138-0e11-4836-b0c9-5201e8e5b6e7-kube-api-access-dzh9k\") pod \"maas-api-cb9fbb88b-56xzv\" (UID: \"b5927138-0e11-4836-b0c9-5201e8e5b6e7\") " pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.779887 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.779892    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5927138-0e11-4836-b0c9-5201e8e5b6e7-maas-api-tls\") pod \"maas-api-cb9fbb88b-56xzv\" (UID: \"b5927138-0e11-4836-b0c9-5201e8e5b6e7\") " pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.780145 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.779922    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5927138-0e11-4836-b0c9-5201e8e5b6e7-openshift-service-ca\") pod \"maas-api-cb9fbb88b-56xzv\" (UID: \"b5927138-0e11-4836-b0c9-5201e8e5b6e7\") " pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.780145 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.780001    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5927138-0e11-4836-b0c9-5201e8e5b6e7-metrics-certs\") pod \"maas-api-cb9fbb88b-56xzv\" (UID: \"b5927138-0e11-4836-b0c9-5201e8e5b6e7\") " pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.880711 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.880678    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5927138-0e11-4836-b0c9-5201e8e5b6e7-openshift-service-ca\") pod \"maas-api-cb9fbb88b-56xzv\" (UID: \"b5927138-0e11-4836-b0c9-5201e8e5b6e7\") " pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.880711 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.880717    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5927138-0e11-4836-b0c9-5201e8e5b6e7-metrics-certs\") pod \"maas-api-cb9fbb88b-56xzv\" (UID: \"b5927138-0e11-4836-b0c9-5201e8e5b6e7\") " pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.880965 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.880803    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzh9k\" (UniqueName: \"kubernetes.io/projected/b5927138-0e11-4836-b0c9-5201e8e5b6e7-kube-api-access-dzh9k\") pod \"maas-api-cb9fbb88b-56xzv\" (UID: \"b5927138-0e11-4836-b0c9-5201e8e5b6e7\") " pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.880965 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.880929    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5927138-0e11-4836-b0c9-5201e8e5b6e7-maas-api-tls\") pod \"maas-api-cb9fbb88b-56xzv\" (UID: \"b5927138-0e11-4836-b0c9-5201e8e5b6e7\") " pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.881481 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.881458    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5927138-0e11-4836-b0c9-5201e8e5b6e7-openshift-service-ca\") pod \"maas-api-cb9fbb88b-56xzv\" (UID: \"b5927138-0e11-4836-b0c9-5201e8e5b6e7\") " pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.883280 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.883257    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5927138-0e11-4836-b0c9-5201e8e5b6e7-metrics-certs\") pod \"maas-api-cb9fbb88b-56xzv\" (UID: \"b5927138-0e11-4836-b0c9-5201e8e5b6e7\") " pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.883373 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.883355    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5927138-0e11-4836-b0c9-5201e8e5b6e7-maas-api-tls\") pod \"maas-api-cb9fbb88b-56xzv\" (UID: \"b5927138-0e11-4836-b0c9-5201e8e5b6e7\") " pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.897938 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.897912    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzh9k\" (UniqueName: \"kubernetes.io/projected/b5927138-0e11-4836-b0c9-5201e8e5b6e7-kube-api-access-dzh9k\") pod \"maas-api-cb9fbb88b-56xzv\" (UID: \"b5927138-0e11-4836-b0c9-5201e8e5b6e7\") " pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:41.972475 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:41.972404    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:42.263749 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:42.263670    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"]
Nov 03 22:02:42.269176 ip-10-0-137-212 kubenswrapper[4454]: W1103 22:02:42.269151    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5927138_0e11_4836_b0c9_5201e8e5b6e7.slice/crio-7551595bb0627471e0722e925d9e4842289756c69dea3164866cb8a891a874aa WatchSource:0}: Error finding container 7551595bb0627471e0722e925d9e4842289756c69dea3164866cb8a891a874aa: Status 404 returned error can't find the container with id 7551595bb0627471e0722e925d9e4842289756c69dea3164866cb8a891a874aa
Nov 03 22:02:42.270519 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:42.270486    4454 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:02:42.878500 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:42.878463    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv" event={"ID":"b5927138-0e11-4836-b0c9-5201e8e5b6e7","Type":"ContainerStarted","Data":"2ce6b0ae0e00eee68205d1d7de5ede0cfd08cfc89d7172c7a4abd54c1889a852"}
Nov 03 22:02:42.878930 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:42.878540    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv" event={"ID":"b5927138-0e11-4836-b0c9-5201e8e5b6e7","Type":"ContainerStarted","Data":"7551595bb0627471e0722e925d9e4842289756c69dea3164866cb8a891a874aa"}
Nov 03 22:02:42.878930 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:42.878652    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:42.916727 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:42.916674    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv" podStartSLOduration=1.604995856 podStartE2EDuration="1.916659914s" totalImagesPullingTime="311.664058ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:02:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:02:42.91595169 +0000 UTC m=+1971.794773957" watchObservedRunningTime="2026-11-03 22:02:42.916659914 +0000 UTC m=+1971.795482198"
Nov 03 22:02:48.888719 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:48.888690    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-cb9fbb88b-56xzv"
Nov 03 22:02:52.036838 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:52.036758    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:02:52.370348 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:02:52.370314    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:02:52.441027 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:52.440991    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:02:52.541207 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:52.541167    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:02:53.713378 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:53.713340    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:02:53.856840 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:53.856805    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:02:56.295412 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:56.295369    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7458978ff4-76djf"]
Nov 03 22:02:56.295885 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:56.295660    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf" podUID="0463c19a-5eba-4083-b2cc-1d186b29b59d" containerName="maas-api" containerID="cri-o://50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea" gracePeriod=30
Nov 03 22:02:57.543312 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.543288    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 22:02:57.634171 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.634089    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-maas-api-tls\") pod \"0463c19a-5eba-4083-b2cc-1d186b29b59d\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") "
Nov 03 22:02:57.634171 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.634140    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0463c19a-5eba-4083-b2cc-1d186b29b59d-kube-api-access-ndjxz\" (UniqueName: \"kubernetes.io/projected/0463c19a-5eba-4083-b2cc-1d186b29b59d-kube-api-access-ndjxz\") pod \"0463c19a-5eba-4083-b2cc-1d186b29b59d\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") "
Nov 03 22:02:57.634365 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.634224    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0463c19a-5eba-4083-b2cc-1d186b29b59d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0463c19a-5eba-4083-b2cc-1d186b29b59d-openshift-service-ca\") pod \"0463c19a-5eba-4083-b2cc-1d186b29b59d\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") "
Nov 03 22:02:57.634365 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.634272    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-metrics-certs\") pod \"0463c19a-5eba-4083-b2cc-1d186b29b59d\" (UID: \"0463c19a-5eba-4083-b2cc-1d186b29b59d\") "
Nov 03 22:02:57.634645 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.634611    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0463c19a-5eba-4083-b2cc-1d186b29b59d-openshift-service-ca" pod "0463c19a-5eba-4083-b2cc-1d186b29b59d" (UID: "0463c19a-5eba-4083-b2cc-1d186b29b59d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:02:57.636314 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.636290    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-metrics-certs" pod "0463c19a-5eba-4083-b2cc-1d186b29b59d" (UID: "0463c19a-5eba-4083-b2cc-1d186b29b59d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:02:57.636412 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.636339    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-maas-api-tls" pod "0463c19a-5eba-4083-b2cc-1d186b29b59d" (UID: "0463c19a-5eba-4083-b2cc-1d186b29b59d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:02:57.636412 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.636364    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0463c19a-5eba-4083-b2cc-1d186b29b59d-kube-api-access-ndjxz" pod "0463c19a-5eba-4083-b2cc-1d186b29b59d" (UID: "0463c19a-5eba-4083-b2cc-1d186b29b59d"). InnerVolumeSpecName "kube-api-access-ndjxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:02:57.735915 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.735879    4454 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0463c19a-5eba-4083-b2cc-1d186b29b59d-openshift-service-ca\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:02:57.735915 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.735909    4454 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-metrics-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:02:57.735915 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.735918    4454 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0463c19a-5eba-4083-b2cc-1d186b29b59d-maas-api-tls\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:02:57.736142 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.735926    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndjxz\" (UniqueName: \"kubernetes.io/projected/0463c19a-5eba-4083-b2cc-1d186b29b59d-kube-api-access-ndjxz\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:02:57.945797 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.945710    4454 generic.go:356] "Generic (PLEG): container finished" podID="0463c19a-5eba-4083-b2cc-1d186b29b59d" containerID="50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea" exitCode=0
Nov 03 22:02:57.945797 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.945755    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf"
Nov 03 22:02:57.945797 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.945787    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf" event={"ID":"0463c19a-5eba-4083-b2cc-1d186b29b59d","Type":"ContainerDied","Data":"50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea"}
Nov 03 22:02:57.946029 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.945825    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7458978ff4-76djf" event={"ID":"0463c19a-5eba-4083-b2cc-1d186b29b59d","Type":"ContainerDied","Data":"3c92df2a4690ae9bcdbbbe9a4966f0f36b0231c7961456f8a786e14f1e68c52f"}
Nov 03 22:02:57.946029 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.945847    4454 scope.go:122] "RemoveContainer" containerID="50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea"
Nov 03 22:02:57.954797 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.954780    4454 scope.go:122] "RemoveContainer" containerID="50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea"
Nov 03 22:02:57.955061 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:02:57.955040    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea\": container with ID starting with 50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea not found: ID does not exist" containerID="50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea"
Nov 03 22:02:57.955127 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.955074    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea"} err="failed to get container status \"50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea\": rpc error: code = NotFound desc = could not find container \"50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea\": container with ID starting with 50122012e2303622e466aa01e1abfb395b8a599e250785b0d495fd6584ce15ea not found: ID does not exist"
Nov 03 22:02:57.991238 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:57.991212    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7458978ff4-76djf"]
Nov 03 22:02:58.002478 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:58.002452    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7458978ff4-76djf"]
Nov 03 22:02:58.459498 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:58.459461    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:02:58.908042 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:58.907992    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:02:59.011954 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:59.011917    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:02:59.228358 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:02:59.228279    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0463c19a-5eba-4083-b2cc-1d186b29b59d" path="/var/lib/kubelet/pods/0463c19a-5eba-4083-b2cc-1d186b29b59d/volumes"
Nov 03 22:03:02.163243 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:02.163202    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:02.267820 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:02.267776    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:03.834863 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.834824    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:03.866378 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.866347    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"]
Nov 03 22:03:03.866970 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.866941    4454 cpu_manager.go:414] "containerMap: removing container" podUID="0463c19a-5eba-4083-b2cc-1d186b29b59d" containerName="maas-api"
Nov 03 22:03:03.866970 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.866969    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0463c19a-5eba-4083-b2cc-1d186b29b59d" containerName="maas-api"
Nov 03 22:03:03.867184 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.867051    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="0463c19a-5eba-4083-b2cc-1d186b29b59d" containerName="maas-api"
Nov 03 22:03:03.871563 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.871545    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"
Nov 03 22:03:03.890372 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.890354    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b14ec9a0\""
Nov 03 22:03:03.894428 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.894404    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"]
Nov 03 22:03:03.937281 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.937238    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:03.938784 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.938766    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"]
Nov 03 22:03:03.942546 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.942527    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"
Nov 03 22:03:03.955958 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.955936    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b14ec9a0-dockercfg-nbmlk\""
Nov 03 22:03:03.961930 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:03.961909    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"]
Nov 03 22:03:04.006530 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.005077    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68bacd31-c520-49ec-80dd-a58b8d51b0ec-plugins-config-volume\") pod \"payload-processing-e2e-ait-b14ec9a0-87558f764-984cm\" (UID: \"68bacd31-c520-49ec-80dd-a58b8d51b0ec\") " pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"
Nov 03 22:03:04.006530 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.005311    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j7r5\" (UniqueName: \"kubernetes.io/projected/68bacd31-c520-49ec-80dd-a58b8d51b0ec-kube-api-access-8j7r5\") pod \"payload-processing-e2e-ait-b14ec9a0-87558f764-984cm\" (UID: \"68bacd31-c520-49ec-80dd-a58b8d51b0ec\") " pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"
Nov 03 22:03:04.106577 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.106545    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba10e77-82d8-447e-be8d-4f5ff82592d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7\" (UID: \"7ba10e77-82d8-447e-be8d-4f5ff82592d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"
Nov 03 22:03:04.106738 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.106610    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xwbr\" (UniqueName: \"kubernetes.io/projected/7ba10e77-82d8-447e-be8d-4f5ff82592d0-kube-api-access-4xwbr\") pod \"payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7\" (UID: \"7ba10e77-82d8-447e-be8d-4f5ff82592d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"
Nov 03 22:03:04.106738 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.106719    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j7r5\" (UniqueName: \"kubernetes.io/projected/68bacd31-c520-49ec-80dd-a58b8d51b0ec-kube-api-access-8j7r5\") pod \"payload-processing-e2e-ait-b14ec9a0-87558f764-984cm\" (UID: \"68bacd31-c520-49ec-80dd-a58b8d51b0ec\") " pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"
Nov 03 22:03:04.106817 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.106746    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68bacd31-c520-49ec-80dd-a58b8d51b0ec-plugins-config-volume\") pod \"payload-processing-e2e-ait-b14ec9a0-87558f764-984cm\" (UID: \"68bacd31-c520-49ec-80dd-a58b8d51b0ec\") " pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"
Nov 03 22:03:04.107463 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.107443    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68bacd31-c520-49ec-80dd-a58b8d51b0ec-plugins-config-volume\") pod \"payload-processing-e2e-ait-b14ec9a0-87558f764-984cm\" (UID: \"68bacd31-c520-49ec-80dd-a58b8d51b0ec\") " pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"
Nov 03 22:03:04.124856 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.124832    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j7r5\" (UniqueName: \"kubernetes.io/projected/68bacd31-c520-49ec-80dd-a58b8d51b0ec-kube-api-access-8j7r5\") pod \"payload-processing-e2e-ait-b14ec9a0-87558f764-984cm\" (UID: \"68bacd31-c520-49ec-80dd-a58b8d51b0ec\") " pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"
Nov 03 22:03:04.183726 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.183696    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"
Nov 03 22:03:04.208329 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.208296    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xwbr\" (UniqueName: \"kubernetes.io/projected/7ba10e77-82d8-447e-be8d-4f5ff82592d0-kube-api-access-4xwbr\") pod \"payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7\" (UID: \"7ba10e77-82d8-447e-be8d-4f5ff82592d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"
Nov 03 22:03:04.208482 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.208462    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba10e77-82d8-447e-be8d-4f5ff82592d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7\" (UID: \"7ba10e77-82d8-447e-be8d-4f5ff82592d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"
Nov 03 22:03:04.209155 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.209129    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba10e77-82d8-447e-be8d-4f5ff82592d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7\" (UID: \"7ba10e77-82d8-447e-be8d-4f5ff82592d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"
Nov 03 22:03:04.226054 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.226028    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xwbr\" (UniqueName: \"kubernetes.io/projected/7ba10e77-82d8-447e-be8d-4f5ff82592d0-kube-api-access-4xwbr\") pod \"payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7\" (UID: \"7ba10e77-82d8-447e-be8d-4f5ff82592d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"
Nov 03 22:03:04.255433 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.255336    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"
Nov 03 22:03:04.474591 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.474442    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"]
Nov 03 22:03:04.479022 ip-10-0-137-212 kubenswrapper[4454]: W1103 22:03:04.478994    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68bacd31_c520_49ec_80dd_a58b8d51b0ec.slice/crio-e5c60ec19615497b08cddf03eac5f5a489eeaf58cecdafe013d8772c2c5a06ac WatchSource:0}: Error finding container e5c60ec19615497b08cddf03eac5f5a489eeaf58cecdafe013d8772c2c5a06ac: Status 404 returned error can't find the container with id e5c60ec19615497b08cddf03eac5f5a489eeaf58cecdafe013d8772c2c5a06ac
Nov 03 22:03:04.551397 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.551360    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"]
Nov 03 22:03:04.556118 ip-10-0-137-212 kubenswrapper[4454]: W1103 22:03:04.556090    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ba10e77_82d8_447e_be8d_4f5ff82592d0.slice/crio-dd9bb27954dc74df3789da6bfa0003e04e2ca6bdb75179ef29ba516bd807f80b WatchSource:0}: Error finding container dd9bb27954dc74df3789da6bfa0003e04e2ca6bdb75179ef29ba516bd807f80b: Status 404 returned error can't find the container with id dd9bb27954dc74df3789da6bfa0003e04e2ca6bdb75179ef29ba516bd807f80b
Nov 03 22:03:04.622708 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.622679    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"]
Nov 03 22:03:04.626315 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.626298    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.639957 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.639938    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b14ec9a0-serving-cert\""
Nov 03 22:03:04.646607 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.646581    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"]
Nov 03 22:03:04.815238 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.815153    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e128434-7130-4505-aeef-1eb9ab98d8ec-openshift-service-ca\") pod \"maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.815238 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.815194    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-metrics-certs\") pod \"maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.815238 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.815228    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-maas-api-tls\") pod \"maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.815478 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.815325    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpdsb\" (UniqueName: \"kubernetes.io/projected/0e128434-7130-4505-aeef-1eb9ab98d8ec-kube-api-access-hpdsb\") pod \"maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.818039 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.818000    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:04.916728 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.916693    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpdsb\" (UniqueName: \"kubernetes.io/projected/0e128434-7130-4505-aeef-1eb9ab98d8ec-kube-api-access-hpdsb\") pod \"maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.917212 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.916796    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e128434-7130-4505-aeef-1eb9ab98d8ec-openshift-service-ca\") pod \"maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.917212 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.916841    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-metrics-certs\") pod \"maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.917212 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.916879    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-maas-api-tls\") pod \"maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.917682 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.917652    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e128434-7130-4505-aeef-1eb9ab98d8ec-openshift-service-ca\") pod \"maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.919336 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.919314    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-maas-api-tls\") pod \"maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.919447 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.919407    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-metrics-certs\") pod \"maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.923849 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.923822    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:04.935005 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.934987    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpdsb\" (UniqueName: \"kubernetes.io/projected/0e128434-7130-4505-aeef-1eb9ab98d8ec-kube-api-access-hpdsb\") pod \"maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.938806 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.938790    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:04.983532 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.982029    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7" event={"ID":"7ba10e77-82d8-447e-be8d-4f5ff82592d0","Type":"ContainerStarted","Data":"d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a"}
Nov 03 22:03:04.983532 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.982090    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7" event={"ID":"7ba10e77-82d8-447e-be8d-4f5ff82592d0","Type":"ContainerStarted","Data":"dd9bb27954dc74df3789da6bfa0003e04e2ca6bdb75179ef29ba516bd807f80b"}
Nov 03 22:03:04.983532 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.982772    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"
Nov 03 22:03:04.984254 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.984231    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm" event={"ID":"68bacd31-c520-49ec-80dd-a58b8d51b0ec","Type":"ContainerStarted","Data":"915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282"}
Nov 03 22:03:04.984386 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.984266    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm" event={"ID":"68bacd31-c520-49ec-80dd-a58b8d51b0ec","Type":"ContainerStarted","Data":"e5c60ec19615497b08cddf03eac5f5a489eeaf58cecdafe013d8772c2c5a06ac"}
Nov 03 22:03:04.984386 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:04.984369    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"
Nov 03 22:03:05.026147 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:05.026087    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7" podStartSLOduration=2.026072297 podStartE2EDuration="2.026072297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:03:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:03:05.024251898 +0000 UTC m=+1993.903074218" watchObservedRunningTime="2026-11-03 22:03:05.026072297 +0000 UTC m=+1993.904894581"
Nov 03 22:03:05.059764 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:05.059718    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm" podStartSLOduration=2.059704308 podStartE2EDuration="2.059704308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:03:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:03:05.058891188 +0000 UTC m=+1993.937713494" watchObservedRunningTime="2026-11-03 22:03:05.059704308 +0000 UTC m=+1993.938526589"
Nov 03 22:03:05.233343 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:05.233305    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"]
Nov 03 22:03:05.237961 ip-10-0-137-212 kubenswrapper[4454]: W1103 22:03:05.237931    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e128434_7130_4505_aeef_1eb9ab98d8ec.slice/crio-d08a8fb8b4ec959df83dc502716b57621f9de2c2b9095d6a04af0f92d86fac06 WatchSource:0}: Error finding container d08a8fb8b4ec959df83dc502716b57621f9de2c2b9095d6a04af0f92d86fac06: Status 404 returned error can't find the container with id d08a8fb8b4ec959df83dc502716b57621f9de2c2b9095d6a04af0f92d86fac06
Nov 03 22:03:05.991212 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:05.991171    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk" event={"ID":"0e128434-7130-4505-aeef-1eb9ab98d8ec","Type":"ContainerStarted","Data":"b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871"}
Nov 03 22:03:05.991623 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:05.991229    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk" event={"ID":"0e128434-7130-4505-aeef-1eb9ab98d8ec","Type":"ContainerStarted","Data":"d08a8fb8b4ec959df83dc502716b57621f9de2c2b9095d6a04af0f92d86fac06"}
Nov 03 22:03:06.028371 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:06.028321    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk" podStartSLOduration=1.741231918 podStartE2EDuration="2.028308721s" totalImagesPullingTime="287.076803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:03:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:03:06.027075065 +0000 UTC m=+1994.905897370" watchObservedRunningTime="2026-11-03 22:03:06.028308721 +0000 UTC m=+1994.907131046"
Nov 03 22:03:06.563171 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:06.563131    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:06.996032 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:06.996005    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:08.256783 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:08.256743    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:08.490532 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:08.490022    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:03:09.989378 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:09.989330    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:11.799207 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:11.799167    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:13.005730 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:13.005698    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:13.692565 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:13.692498    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:15.543363 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:15.543319    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:15.994543 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:15.994497    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"
Nov 03 22:03:16.999765 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:16.999728    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"
Nov 03 22:03:17.257403 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:17.257313    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:19.132832 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:19.132798    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:20.891224 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:20.891190    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:22.371462 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:03:22.371428    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:03:22.671637 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:22.671541    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:24.487937 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:24.487900    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:26.172084 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:26.172044    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:27.942384 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:27.942326    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:28.348603 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:28.348564    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"]
Nov 03 22:03:28.373697 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:28.373661    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"]
Nov 03 22:03:28.519367 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:28.519318    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:28.636311 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:28.636223    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:29.371791 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.371752    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"]
Nov 03 22:03:29.372258 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.371974    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk" podUID="0e128434-7130-4505-aeef-1eb9ab98d8ec" containerName="maas-api" containerID="cri-o://b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871" gracePeriod=30
Nov 03 22:03:29.434158 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.434117    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:29.637967 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.637886    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:29.762774 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.762736    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-maas-api-tls\") pod \"0e128434-7130-4505-aeef-1eb9ab98d8ec\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") "
Nov 03 22:03:29.762964 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.762797    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-metrics-certs\") pod \"0e128434-7130-4505-aeef-1eb9ab98d8ec\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") "
Nov 03 22:03:29.762964 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.762821    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e128434-7130-4505-aeef-1eb9ab98d8ec-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e128434-7130-4505-aeef-1eb9ab98d8ec-openshift-service-ca\") pod \"0e128434-7130-4505-aeef-1eb9ab98d8ec\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") "
Nov 03 22:03:29.762964 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.762892    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e128434-7130-4505-aeef-1eb9ab98d8ec-kube-api-access-hpdsb\" (UniqueName: \"kubernetes.io/projected/0e128434-7130-4505-aeef-1eb9ab98d8ec-kube-api-access-hpdsb\") pod \"0e128434-7130-4505-aeef-1eb9ab98d8ec\" (UID: \"0e128434-7130-4505-aeef-1eb9ab98d8ec\") "
Nov 03 22:03:29.763291 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.763262    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e128434-7130-4505-aeef-1eb9ab98d8ec-openshift-service-ca" pod "0e128434-7130-4505-aeef-1eb9ab98d8ec" (UID: "0e128434-7130-4505-aeef-1eb9ab98d8ec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:03:29.765178 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.765131    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e128434-7130-4505-aeef-1eb9ab98d8ec-kube-api-access-hpdsb" pod "0e128434-7130-4505-aeef-1eb9ab98d8ec" (UID: "0e128434-7130-4505-aeef-1eb9ab98d8ec"). InnerVolumeSpecName "kube-api-access-hpdsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:03:29.765310 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.765242    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-maas-api-tls" pod "0e128434-7130-4505-aeef-1eb9ab98d8ec" (UID: "0e128434-7130-4505-aeef-1eb9ab98d8ec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:03:29.765310 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.765271    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-metrics-certs" pod "0e128434-7130-4505-aeef-1eb9ab98d8ec" (UID: "0e128434-7130-4505-aeef-1eb9ab98d8ec"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:03:29.864037 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.863999    4454 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-maas-api-tls\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:03:29.864037 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.864028    4454 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e128434-7130-4505-aeef-1eb9ab98d8ec-metrics-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:03:29.864037 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.864043    4454 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e128434-7130-4505-aeef-1eb9ab98d8ec-openshift-service-ca\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:03:29.864279 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.864058    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpdsb\" (UniqueName: \"kubernetes.io/projected/0e128434-7130-4505-aeef-1eb9ab98d8ec-kube-api-access-hpdsb\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:03:29.939651 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:29.939560    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:30.039905 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:30.039869    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:30.101182 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:30.101148    4454 generic.go:356] "Generic (PLEG): container finished" podID="0e128434-7130-4505-aeef-1eb9ab98d8ec" containerID="b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871" exitCode=0
Nov 03 22:03:30.101395 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:30.101219    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"
Nov 03 22:03:30.101395 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:30.101252    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk" event={"ID":"0e128434-7130-4505-aeef-1eb9ab98d8ec","Type":"ContainerDied","Data":"b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871"}
Nov 03 22:03:30.101395 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:30.101338    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk" event={"ID":"0e128434-7130-4505-aeef-1eb9ab98d8ec","Type":"ContainerDied","Data":"d08a8fb8b4ec959df83dc502716b57621f9de2c2b9095d6a04af0f92d86fac06"}
Nov 03 22:03:30.101395 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:30.101371    4454 scope.go:122] "RemoveContainer" containerID="b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871"
Nov 03 22:03:30.117960 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:30.112667    4454 scope.go:122] "RemoveContainer" containerID="b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871"
Nov 03 22:03:30.118375 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:03:30.118340    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871\": container with ID starting with b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871 not found: ID does not exist" containerID="b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871"
Nov 03 22:03:30.118488 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:30.118385    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871"} err="failed to get container status \"b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871\": rpc error: code = NotFound desc = could not find container \"b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871\": container with ID starting with b74a0ef28975e46f98eb34aaee21471490f4ef137862bf78c6c066ea47fdb871 not found: ID does not exist"
Nov 03 22:03:30.151573 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:30.151536    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"]
Nov 03 22:03:30.163100 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:30.163076    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b14ec9a0-68cff875fd-c8xvk"]
Nov 03 22:03:31.228692 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:31.228664    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e128434-7130-4505-aeef-1eb9ab98d8ec" path="/var/lib/kubelet/pods/0e128434-7130-4505-aeef-1eb9ab98d8ec/volumes"
Nov 03 22:03:31.612839 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:31.612799    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:03:33.349560 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.349522    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm" podUID="68bacd31-c520-49ec-80dd-a58b8d51b0ec" containerName="payload-processing" containerID="cri-o://915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282" gracePeriod=25
Nov 03 22:03:33.374969 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.374935    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7" podUID="7ba10e77-82d8-447e-be8d-4f5ff82592d0" containerName="payload-pre-processing" containerID="cri-o://d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a" gracePeriod=25
Nov 03 22:03:33.619420 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.619394    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"
Nov 03 22:03:33.641780 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.641756    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"
Nov 03 22:03:33.697720 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.697686    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68bacd31-c520-49ec-80dd-a58b8d51b0ec-kube-api-access-8j7r5\" (UniqueName: \"kubernetes.io/projected/68bacd31-c520-49ec-80dd-a58b8d51b0ec-kube-api-access-8j7r5\") pod \"68bacd31-c520-49ec-80dd-a58b8d51b0ec\" (UID: \"68bacd31-c520-49ec-80dd-a58b8d51b0ec\") "
Nov 03 22:03:33.697889 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.697805    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68bacd31-c520-49ec-80dd-a58b8d51b0ec-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68bacd31-c520-49ec-80dd-a58b8d51b0ec-plugins-config-volume\") pod \"68bacd31-c520-49ec-80dd-a58b8d51b0ec\" (UID: \"68bacd31-c520-49ec-80dd-a58b8d51b0ec\") "
Nov 03 22:03:33.697889 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.697879    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ba10e77-82d8-447e-be8d-4f5ff82592d0-kube-api-access-4xwbr\" (UniqueName: \"kubernetes.io/projected/7ba10e77-82d8-447e-be8d-4f5ff82592d0-kube-api-access-4xwbr\") pod \"7ba10e77-82d8-447e-be8d-4f5ff82592d0\" (UID: \"7ba10e77-82d8-447e-be8d-4f5ff82592d0\") "
Nov 03 22:03:33.697990 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.697921    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ba10e77-82d8-447e-be8d-4f5ff82592d0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba10e77-82d8-447e-be8d-4f5ff82592d0-plugins-config-volume\") pod \"7ba10e77-82d8-447e-be8d-4f5ff82592d0\" (UID: \"7ba10e77-82d8-447e-be8d-4f5ff82592d0\") "
Nov 03 22:03:33.698440 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.698408    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68bacd31-c520-49ec-80dd-a58b8d51b0ec-plugins-config-volume" pod "68bacd31-c520-49ec-80dd-a58b8d51b0ec" (UID: "68bacd31-c520-49ec-80dd-a58b8d51b0ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:03:33.698585 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.698460    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ba10e77-82d8-447e-be8d-4f5ff82592d0-plugins-config-volume" pod "7ba10e77-82d8-447e-be8d-4f5ff82592d0" (UID: "7ba10e77-82d8-447e-be8d-4f5ff82592d0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:03:33.700031 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.700004    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68bacd31-c520-49ec-80dd-a58b8d51b0ec-kube-api-access-8j7r5" pod "68bacd31-c520-49ec-80dd-a58b8d51b0ec" (UID: "68bacd31-c520-49ec-80dd-a58b8d51b0ec"). InnerVolumeSpecName "kube-api-access-8j7r5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:03:33.700550 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.700485    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ba10e77-82d8-447e-be8d-4f5ff82592d0-kube-api-access-4xwbr" pod "7ba10e77-82d8-447e-be8d-4f5ff82592d0" (UID: "7ba10e77-82d8-447e-be8d-4f5ff82592d0"). InnerVolumeSpecName "kube-api-access-4xwbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:03:33.799165 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.799123    4454 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68bacd31-c520-49ec-80dd-a58b8d51b0ec-plugins-config-volume\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:03:33.799165 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.799159    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xwbr\" (UniqueName: \"kubernetes.io/projected/7ba10e77-82d8-447e-be8d-4f5ff82592d0-kube-api-access-4xwbr\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:03:33.799165 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.799171    4454 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba10e77-82d8-447e-be8d-4f5ff82592d0-plugins-config-volume\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:03:33.799403 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:33.799183    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8j7r5\" (UniqueName: \"kubernetes.io/projected/68bacd31-c520-49ec-80dd-a58b8d51b0ec-kube-api-access-8j7r5\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:03:34.121031 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.120991    4454 generic.go:356] "Generic (PLEG): container finished" podID="7ba10e77-82d8-447e-be8d-4f5ff82592d0" containerID="d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a" exitCode=0
Nov 03 22:03:34.121231 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.121044    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"
Nov 03 22:03:34.121231 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.121070    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7" event={"ID":"7ba10e77-82d8-447e-be8d-4f5ff82592d0","Type":"ContainerDied","Data":"d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a"}
Nov 03 22:03:34.121231 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.121122    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7" event={"ID":"7ba10e77-82d8-447e-be8d-4f5ff82592d0","Type":"ContainerDied","Data":"dd9bb27954dc74df3789da6bfa0003e04e2ca6bdb75179ef29ba516bd807f80b"}
Nov 03 22:03:34.121231 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.121146    4454 scope.go:122] "RemoveContainer" containerID="d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a"
Nov 03 22:03:34.122406 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.122384    4454 generic.go:356] "Generic (PLEG): container finished" podID="68bacd31-c520-49ec-80dd-a58b8d51b0ec" containerID="915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282" exitCode=0
Nov 03 22:03:34.122502 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.122433    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"
Nov 03 22:03:34.122502 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.122472    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm" event={"ID":"68bacd31-c520-49ec-80dd-a58b8d51b0ec","Type":"ContainerDied","Data":"915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282"}
Nov 03 22:03:34.122639 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.122538    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm" event={"ID":"68bacd31-c520-49ec-80dd-a58b8d51b0ec","Type":"ContainerDied","Data":"e5c60ec19615497b08cddf03eac5f5a489eeaf58cecdafe013d8772c2c5a06ac"}
Nov 03 22:03:34.129854 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.129835    4454 scope.go:122] "RemoveContainer" containerID="d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a"
Nov 03 22:03:34.130193 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:03:34.130169    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a\": container with ID starting with d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a not found: ID does not exist" containerID="d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a"
Nov 03 22:03:34.130282 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.130200    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a"} err="failed to get container status \"d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a\": rpc error: code = NotFound desc = could not find container \"d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a\": container with ID starting with d1e874b333c218ecffd6b822b22f4cd669ba94b4cf12f8b5fc856ef356cb144a not found: ID does not exist"
Nov 03 22:03:34.130282 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.130221    4454 scope.go:122] "RemoveContainer" containerID="915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282"
Nov 03 22:03:34.138439 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.138420    4454 scope.go:122] "RemoveContainer" containerID="915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282"
Nov 03 22:03:34.138704 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:03:34.138684    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282\": container with ID starting with 915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282 not found: ID does not exist" containerID="915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282"
Nov 03 22:03:34.138755 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.138712    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282"} err="failed to get container status \"915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282\": rpc error: code = NotFound desc = could not find container \"915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282\": container with ID starting with 915d010359ce3e87cbed1818b76e5e02ce479755e7320006e05bbb057bcb6282 not found: ID does not exist"
Nov 03 22:03:34.167745 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.167718    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"]
Nov 03 22:03:34.178908 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.178876    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b14ec9a0-87558f764-984cm"]
Nov 03 22:03:34.209658 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.209617    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"]
Nov 03 22:03:34.212945 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.212915    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b14ec9a0-8594776f87-wn6t7"]
Nov 03 22:03:34.272484 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:34.272452    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:35.228808 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.228776    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68bacd31-c520-49ec-80dd-a58b8d51b0ec" path="/var/lib/kubelet/pods/68bacd31-c520-49ec-80dd-a58b8d51b0ec/volumes"
Nov 03 22:03:35.229207 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.229190    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ba10e77-82d8-447e-be8d-4f5ff82592d0" path="/var/lib/kubelet/pods/7ba10e77-82d8-447e-be8d-4f5ff82592d0/volumes"
Nov 03 22:03:35.505199 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.505102    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:35.896984 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.896460    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"]
Nov 03 22:03:35.897162 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.897078    4454 cpu_manager.go:414] "containerMap: removing container" podUID="68bacd31-c520-49ec-80dd-a58b8d51b0ec" containerName="payload-processing"
Nov 03 22:03:35.897162 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.897100    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68bacd31-c520-49ec-80dd-a58b8d51b0ec" containerName="payload-processing"
Nov 03 22:03:35.897162 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.897125    4454 cpu_manager.go:414] "containerMap: removing container" podUID="7ba10e77-82d8-447e-be8d-4f5ff82592d0" containerName="payload-pre-processing"
Nov 03 22:03:35.897162 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.897133    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ba10e77-82d8-447e-be8d-4f5ff82592d0" containerName="payload-pre-processing"
Nov 03 22:03:35.897162 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.897148    4454 cpu_manager.go:414] "containerMap: removing container" podUID="0e128434-7130-4505-aeef-1eb9ab98d8ec" containerName="maas-api"
Nov 03 22:03:35.897162 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.897156    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e128434-7130-4505-aeef-1eb9ab98d8ec" containerName="maas-api"
Nov 03 22:03:35.897388 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.897236    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e128434-7130-4505-aeef-1eb9ab98d8ec" containerName="maas-api"
Nov 03 22:03:35.897388 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.897252    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="68bacd31-c520-49ec-80dd-a58b8d51b0ec" containerName="payload-processing"
Nov 03 22:03:35.897388 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.897264    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ba10e77-82d8-447e-be8d-4f5ff82592d0" containerName="payload-pre-processing"
Nov 03 22:03:35.902649 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.902624    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:35.916837 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.916810    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-6a697e-openshift-default-dockercfg-qsxdd\""
Nov 03 22:03:35.933596 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.933561    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"]
Nov 03 22:03:35.949797 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:35.949761    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:36.029286 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.025067    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-istio-token\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.029628 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.029588    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-certs\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.029751 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.029636    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96914100-0306-4d16-b413-97d487e4f6ef-istiod-ca-cert\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.029804 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.029740    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-envoy\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.029839 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.029816    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-socket\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.029887 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.029863    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w45zp\" (UniqueName: \"kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-kube-api-access-w45zp\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.029962 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.029925    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96914100-0306-4d16-b413-97d487e4f6ef-istio-podinfo\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.030092 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.029970    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-credential-socket\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.030092 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.030003    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-data\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.130585 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.130546    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96914100-0306-4d16-b413-97d487e4f6ef-istio-podinfo\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.130765 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.130595    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-credential-socket\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.130765 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.130620    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-data\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.130765 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.130714    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-istio-token\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.130765 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.130741    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-certs\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.130999 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.130768    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96914100-0306-4d16-b413-97d487e4f6ef-istiod-ca-cert\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.130999 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.130793    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-envoy\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.130999 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.130835    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-socket\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.130999 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.130874    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w45zp\" (UniqueName: \"kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-kube-api-access-w45zp\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.131222 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.131031    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-credential-socket\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.131222 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.131065    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-data\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.131222 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.131145    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-certs\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.131544 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.131465    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-socket\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.131810 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.131788    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96914100-0306-4d16-b413-97d487e4f6ef-istiod-ca-cert\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.133237 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.133208    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-envoy\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.133341 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.133323    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96914100-0306-4d16-b413-97d487e4f6ef-istio-podinfo\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.149131 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.149069    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w45zp\" (UniqueName: \"kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-kube-api-access-w45zp\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.149216 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.149134    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-istio-token\") pod \"e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") " pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.151149 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.151123    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:36.220495 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.220463    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:36.527841 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.527753    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"]
Nov 03 22:03:36.532929 ip-10-0-137-212 kubenswrapper[4454]: W1103 22:03:36.532904    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96914100_0306_4d16_b413_97d487e4f6ef.slice/crio-a3b4e784d2803e13ec5d699a800218080ad97538bc025ab235f579bea801b499 WatchSource:0}: Error finding container a3b4e784d2803e13ec5d699a800218080ad97538bc025ab235f579bea801b499: Status 404 returned error can't find the container with id a3b4e784d2803e13ec5d699a800218080ad97538bc025ab235f579bea801b499
Nov 03 22:03:36.535018 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.534986    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 22:03:36.535085 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.535059    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 22:03:36.535129 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:36.535088    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 22:03:37.141304 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:37.141266    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" event={"ID":"96914100-0306-4d16-b413-97d487e4f6ef","Type":"ContainerStarted","Data":"b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80"}
Nov 03 22:03:37.141489 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:37.141328    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" event={"ID":"96914100-0306-4d16-b413-97d487e4f6ef","Type":"ContainerStarted","Data":"a3b4e784d2803e13ec5d699a800218080ad97538bc025ab235f579bea801b499"}
Nov 03 22:03:37.201194 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:37.201138    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" podStartSLOduration=2.201124703 podStartE2EDuration="2.201124703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:03:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:03:37.190014442 +0000 UTC m=+2026.068836727" watchObservedRunningTime="2026-11-03 22:03:37.201124703 +0000 UTC m=+2026.079946988"
Nov 03 22:03:37.221047 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:37.221017    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:37.228159 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:37.228139    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:38.146365 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:38.146335    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:38.147317 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:38.147297    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:38.932295 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:38.932260    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:39.519273 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:39.519231    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:39.638179 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:39.638139    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:41.019094 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.019059    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"]
Nov 03 22:03:41.022966 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.022945    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.038723 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.038702    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-6a697e-serving-cert\""
Nov 03 22:03:41.049868 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.049841    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"]
Nov 03 22:03:41.109460 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.109426    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:41.178570 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.178540    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv25t\" (UniqueName: \"kubernetes.io/projected/bed92553-a8b7-42c0-8309-10606ae1d416-kube-api-access-qv25t\") pod \"maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.178738 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.178583    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed92553-a8b7-42c0-8309-10606ae1d416-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.178781 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.178743    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.178817 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.178783    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-metrics-certs\") pod \"maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.209154 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.209127    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:41.279720 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.279544    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.279720 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.279611    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-metrics-certs\") pod \"maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.279720 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.279683    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv25t\" (UniqueName: \"kubernetes.io/projected/bed92553-a8b7-42c0-8309-10606ae1d416-kube-api-access-qv25t\") pod \"maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.280215 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.279731    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed92553-a8b7-42c0-8309-10606ae1d416-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.280838 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.280811    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed92553-a8b7-42c0-8309-10606ae1d416-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.283459 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.283431    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.284159 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.284139    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-metrics-certs\") pod \"maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.299114 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.299090    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv25t\" (UniqueName: \"kubernetes.io/projected/bed92553-a8b7-42c0-8309-10606ae1d416-kube-api-access-qv25t\") pod \"maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.308872 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.308847    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:41.344315 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.343467    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:41.678720 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:41.678682    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"]
Nov 03 22:03:41.684087 ip-10-0-137-212 kubenswrapper[4454]: W1103 22:03:41.684057    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbed92553_a8b7_42c0_8309_10606ae1d416.slice/crio-89b4d31333dd58ce82da0be0cd841cf831bd01e8b6828c94c477c17be1c82f0e WatchSource:0}: Error finding container 89b4d31333dd58ce82da0be0cd841cf831bd01e8b6828c94c477c17be1c82f0e: Status 404 returned error can't find the container with id 89b4d31333dd58ce82da0be0cd841cf831bd01e8b6828c94c477c17be1c82f0e
Nov 03 22:03:42.169217 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:42.169172    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs" event={"ID":"bed92553-a8b7-42c0-8309-10606ae1d416","Type":"ContainerStarted","Data":"81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994"}
Nov 03 22:03:42.169715 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:42.169237    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs" event={"ID":"bed92553-a8b7-42c0-8309-10606ae1d416","Type":"ContainerStarted","Data":"89b4d31333dd58ce82da0be0cd841cf831bd01e8b6828c94c477c17be1c82f0e"}
Nov 03 22:03:42.169715 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:42.169387    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:42.206851 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:42.206789    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs" podStartSLOduration=1.9000708899999998 podStartE2EDuration="2.206772784s" totalImagesPullingTime="306.701894ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:03:42.205325553 +0000 UTC m=+2031.084147840" watchObservedRunningTime="2026-11-03 22:03:42.206772784 +0000 UTC m=+2031.085595069"
Nov 03 22:03:42.847993 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:42.847953    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:44.566036 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:44.566002    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:44.674530 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:44.673699    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:03:46.285484 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:46.285446    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:47.959745 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:47.959711    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:48.189760 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:48.189729    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:03:49.686427 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:49.686393    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:51.470498 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:51.470410    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:52.372610 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:03:52.372569    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:03:53.158964 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:53.158926    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:54.115859 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:54.114987    4454 patch_prober.go:29] interesting pod/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": read tcp 10.133.0.2:54210->10.133.0.88:15021: read: connection reset by peer" start-of-body=
Nov 03 22:03:54.115859 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:54.115065    4454 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": read tcp 10.133.0.2:54210->10.133.0.88:15021: read: connection reset by peer"
Nov 03 22:03:54.232695 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:54.232662    4454 generic.go:356] "Generic (PLEG): container finished" podID="96914100-0306-4d16-b413-97d487e4f6ef" containerID="b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80" exitCode=0
Nov 03 22:03:54.233067 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:54.232738    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" event={"ID":"96914100-0306-4d16-b413-97d487e4f6ef","Type":"ContainerDied","Data":"b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80"}
Nov 03 22:03:54.233138 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:54.233123    4454 scope.go:122] "RemoveContainer" containerID="b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80"
Nov 03 22:03:54.234379 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:54.234353    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 22:03:54.234452 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:54.234423    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 22:03:54.234495 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:54.234456    4454 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Nov 03 22:03:54.417248 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:54.417148    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:54.850957 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:54.850923    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:55.238631 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:55.238553    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" event={"ID":"96914100-0306-4d16-b413-97d487e4f6ef","Type":"ContainerStarted","Data":"aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4"}
Nov 03 22:03:56.221138 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:56.221091    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:03:56.221404 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:56.221381    4454 patch_prober.go:29] interesting pod/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Nov 03 22:03:56.221456 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:56.221430    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Nov 03 22:03:56.576613 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:56.569989    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:57.221912 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:57.221877    4454 patch_prober.go:29] interesting pod/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Nov 03 22:03:57.222187 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:57.221928    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Nov 03 22:03:58.221570 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:58.221538    4454 patch_prober.go:29] interesting pod/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Nov 03 22:03:58.221988 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:58.221604    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Nov 03 22:03:58.250264 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:58.250228    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:03:59.221758 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:59.221722    4454 patch_prober.go:29] interesting pod/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Nov 03 22:03:59.222203 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:59.221787    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Nov 03 22:03:59.949693 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:03:59.949658    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:00.221675 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:00.221580    4454 patch_prober.go:29] interesting pod/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Nov 03 22:04:00.221675 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:00.221649    4454 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Nov 03 22:04:00.898576 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:00.898539    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:00.999059 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:00.999020    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:01.228682 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:01.228610    4454 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:04:01.229053 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:01.228880    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:04:01.229485 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:01.229467    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:04:01.660403 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:01.660361    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:03.365006 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:03.364969    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:05.054162 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:05.054131    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:06.753908 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:06.753870    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:08.459214 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:08.459176    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:10.157829 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:10.157789    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:11.855697 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:11.855660    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:13.562208 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:13.562166    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:15.251233 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:15.251197    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:16.869147 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:16.869114    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:18.639480 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:18.639441    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:20.273292 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:20.273259    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:22.099744 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:22.099702    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:22.371947 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:04:22.371870    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:04:23.645916 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:23.645882    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:25.344254 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:25.344216    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:26.936490 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:26.936452    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:28.664897 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:28.664860    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:30.357886 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:30.357839    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:32.040624 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:32.040591    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:33.764119 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:33.764082    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:35.473114 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:35.473079    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:37.142065 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:37.142022    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:38.856062 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:38.856026    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:39.042977 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:39.042935    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:39.143101 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:39.143009    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:40.558277 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:40.558238    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:42.290592 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:42.290555    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:44.060173 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:44.060134    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:45.842850 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:45.842815    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:47.460817 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:47.460781    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:49.248946 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:49.248908    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:50.864458 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:50.864422    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:51.419642 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:51.417527    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 22:04:51.424821 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:51.424801    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 22:04:52.370772 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:04:52.370731    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:04:52.609200 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:52.609164    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:54.252440 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:54.252403    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:55.903344 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:55.903311    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:57.564780 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:57.564735    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:04:59.364705 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:04:59.364664    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:01.152644 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:01.152595    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:02.833406 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:02.833367    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:04.470390 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:04.470314    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:06.217955 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:06.217915    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:07.775426 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:07.775384    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:09.457628 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:09.457591    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:11.264911 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:11.264870    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:13.058009 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:13.057973    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:14.871266 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:14.871227    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:16.675435 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:16.675398    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:18.265347 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:18.265303    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:20.066334 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:20.066298    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:21.662300 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:21.662260    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:22.371365 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:05:22.371320    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:05:23.474013 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:23.473980    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:25.002186 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:25.002143    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:25.102402 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:25.102365    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:25.202378 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:25.202345    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:31.615122 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:31.615089    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"]
Nov 03 22:05:31.615700 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:31.615465    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs" podUID="bed92553-a8b7-42c0-8309-10606ae1d416" containerName="maas-api" containerID="cri-o://81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994" gracePeriod=30
Nov 03 22:05:31.708057 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:31.708021    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:31.860628 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:31.860603    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:05:31.939978 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:31.939894    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-metrics-certs\") pod \"bed92553-a8b7-42c0-8309-10606ae1d416\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") "
Nov 03 22:05:31.940151 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:31.939986    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bed92553-a8b7-42c0-8309-10606ae1d416-kube-api-access-qv25t\" (UniqueName: \"kubernetes.io/projected/bed92553-a8b7-42c0-8309-10606ae1d416-kube-api-access-qv25t\") pod \"bed92553-a8b7-42c0-8309-10606ae1d416\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") "
Nov 03 22:05:31.940151 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:31.940086    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bed92553-a8b7-42c0-8309-10606ae1d416-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed92553-a8b7-42c0-8309-10606ae1d416-openshift-service-ca\") pod \"bed92553-a8b7-42c0-8309-10606ae1d416\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") "
Nov 03 22:05:31.940151 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:31.940128    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-maas-api-tls\") pod \"bed92553-a8b7-42c0-8309-10606ae1d416\" (UID: \"bed92553-a8b7-42c0-8309-10606ae1d416\") "
Nov 03 22:05:31.940485 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:31.940459    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bed92553-a8b7-42c0-8309-10606ae1d416-openshift-service-ca" pod "bed92553-a8b7-42c0-8309-10606ae1d416" (UID: "bed92553-a8b7-42c0-8309-10606ae1d416"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:05:31.942085 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:31.942059    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bed92553-a8b7-42c0-8309-10606ae1d416-kube-api-access-qv25t" pod "bed92553-a8b7-42c0-8309-10606ae1d416" (UID: "bed92553-a8b7-42c0-8309-10606ae1d416"). InnerVolumeSpecName "kube-api-access-qv25t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:05:31.942190 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:31.942088    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-metrics-certs" pod "bed92553-a8b7-42c0-8309-10606ae1d416" (UID: "bed92553-a8b7-42c0-8309-10606ae1d416"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:05:31.942190 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:31.942105    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-maas-api-tls" pod "bed92553-a8b7-42c0-8309-10606ae1d416" (UID: "bed92553-a8b7-42c0-8309-10606ae1d416"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:05:32.051413 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.048860    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv25t\" (UniqueName: \"kubernetes.io/projected/bed92553-a8b7-42c0-8309-10606ae1d416-kube-api-access-qv25t\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:32.051413 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.048910    4454 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed92553-a8b7-42c0-8309-10606ae1d416-openshift-service-ca\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:32.051413 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.048935    4454 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-maas-api-tls\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:32.051413 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.048956    4454 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bed92553-a8b7-42c0-8309-10606ae1d416-metrics-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:32.179650 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.179612    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:32.307666 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.307581    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:32.682821 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.682783    4454 generic.go:356] "Generic (PLEG): container finished" podID="bed92553-a8b7-42c0-8309-10606ae1d416" containerID="81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994" exitCode=0
Nov 03 22:05:32.682821 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.682826    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs" event={"ID":"bed92553-a8b7-42c0-8309-10606ae1d416","Type":"ContainerDied","Data":"81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994"}
Nov 03 22:05:32.683383 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.682838    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"
Nov 03 22:05:32.683383 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.682871    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs" event={"ID":"bed92553-a8b7-42c0-8309-10606ae1d416","Type":"ContainerDied","Data":"89b4d31333dd58ce82da0be0cd841cf831bd01e8b6828c94c477c17be1c82f0e"}
Nov 03 22:05:32.683383 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.682893    4454 scope.go:122] "RemoveContainer" containerID="81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994"
Nov 03 22:05:32.692558 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.692538    4454 scope.go:122] "RemoveContainer" containerID="81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994"
Nov 03 22:05:32.692840 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:05:32.692823    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994\": container with ID starting with 81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994 not found: ID does not exist" containerID="81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994"
Nov 03 22:05:32.692886 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.692850    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994"} err="failed to get container status \"81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994\": rpc error: code = NotFound desc = could not find container \"81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994\": container with ID starting with 81255105b3730f1c494508a461eb1d4fa269d830dd2018f2d28a3947f9483994 not found: ID does not exist"
Nov 03 22:05:32.729006 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.728971    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"]
Nov 03 22:05:32.740312 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.740277    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-6a697e-8f6d7ff9b-72nhs"]
Nov 03 22:05:32.836389 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:32.836344    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:33.228400 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:33.228370    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bed92553-a8b7-42c0-8309-10606ae1d416" path="/var/lib/kubelet/pods/bed92553-a8b7-42c0-8309-10606ae1d416/volumes"
Nov 03 22:05:34.881345 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:34.881308    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:05:38.722985 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:38.722949    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:38.854554 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:38.854484    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:39.023796 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:39.023709    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"]
Nov 03 22:05:39.023999 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:39.023967    4454 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy" containerID="cri-o://aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4" gracePeriod=30
Nov 03 22:05:39.067871 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:39.067839    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:39.176527 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:39.176480    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:05:41.681477 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:41.681442    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:05:44.534793 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.534766    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:05:44.627620 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.627571    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-socket\") pod \"96914100-0306-4d16-b413-97d487e4f6ef\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") "
Nov 03 22:05:44.627953 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.627914    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-credential-socket\") pod \"96914100-0306-4d16-b413-97d487e4f6ef\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") "
Nov 03 22:05:44.628174 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.628140    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96914100-0306-4d16-b413-97d487e4f6ef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96914100-0306-4d16-b413-97d487e4f6ef-istiod-ca-cert\") pod \"96914100-0306-4d16-b413-97d487e4f6ef\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") "
Nov 03 22:05:44.628374 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.628356    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-envoy\") pod \"96914100-0306-4d16-b413-97d487e4f6ef\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") "
Nov 03 22:05:44.628543 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.628527    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/96914100-0306-4d16-b413-97d487e4f6ef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96914100-0306-4d16-b413-97d487e4f6ef-istio-podinfo\") pod \"96914100-0306-4d16-b413-97d487e4f6ef\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") "
Nov 03 22:05:44.628701 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.628684    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-kube-api-access-w45zp\" (UniqueName: \"kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-kube-api-access-w45zp\") pod \"96914100-0306-4d16-b413-97d487e4f6ef\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") "
Nov 03 22:05:44.628874 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.628848    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-istio-token\" (UniqueName: \"kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-istio-token\") pod \"96914100-0306-4d16-b413-97d487e4f6ef\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") "
Nov 03 22:05:44.629042 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.629013    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-certs\") pod \"96914100-0306-4d16-b413-97d487e4f6ef\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") "
Nov 03 22:05:44.631191 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.631168    4454 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-data\") pod \"96914100-0306-4d16-b413-97d487e4f6ef\" (UID: \"96914100-0306-4d16-b413-97d487e4f6ef\") "
Nov 03 22:05:44.631817 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.629442    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-certs" pod "96914100-0306-4d16-b413-97d487e4f6ef" (UID: "96914100-0306-4d16-b413-97d487e4f6ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:05:44.631817 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.631780    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-socket" pod "96914100-0306-4d16-b413-97d487e4f6ef" (UID: "96914100-0306-4d16-b413-97d487e4f6ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:05:44.631817 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.629728    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-credential-socket" pod "96914100-0306-4d16-b413-97d487e4f6ef" (UID: "96914100-0306-4d16-b413-97d487e4f6ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:05:44.632060 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.630073    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96914100-0306-4d16-b413-97d487e4f6ef-istiod-ca-cert" pod "96914100-0306-4d16-b413-97d487e4f6ef" (UID: "96914100-0306-4d16-b413-97d487e4f6ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:05:44.632060 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.631682    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-data" pod "96914100-0306-4d16-b413-97d487e4f6ef" (UID: "96914100-0306-4d16-b413-97d487e4f6ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:05:44.632622 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.632605    4454 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-data\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:44.632760 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.632747    4454 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:44.632906 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.632865    4454 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-credential-socket\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:44.633029 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.633015    4454 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96914100-0306-4d16-b413-97d487e4f6ef-istiod-ca-cert\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:44.633140 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.633128    4454 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-workload-certs\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:44.634829 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.634809    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-kube-api-access-w45zp" pod "96914100-0306-4d16-b413-97d487e4f6ef" (UID: "96914100-0306-4d16-b413-97d487e4f6ef"). InnerVolumeSpecName "kube-api-access-w45zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:05:44.634971 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.634826    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-envoy" pod "96914100-0306-4d16-b413-97d487e4f6ef" (UID: "96914100-0306-4d16-b413-97d487e4f6ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:05:44.636793 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.636759    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-istio-token" pod "96914100-0306-4d16-b413-97d487e4f6ef" (UID: "96914100-0306-4d16-b413-97d487e4f6ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:05:44.636882 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.636768    4454 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96914100-0306-4d16-b413-97d487e4f6ef-istio-podinfo" pod "96914100-0306-4d16-b413-97d487e4f6ef" (UID: "96914100-0306-4d16-b413-97d487e4f6ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:05:44.733873 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.733835    4454 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96914100-0306-4d16-b413-97d487e4f6ef-istio-envoy\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:44.733873 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.733867    4454 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96914100-0306-4d16-b413-97d487e4f6ef-istio-podinfo\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:44.733873 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.733878    4454 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w45zp\" (UniqueName: \"kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-kube-api-access-w45zp\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:44.734122 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.733887    4454 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96914100-0306-4d16-b413-97d487e4f6ef-istio-token\") on node \"ip-10-0-137-212.ec2.internal\" DevicePath \"\""
Nov 03 22:05:44.744318 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.744289    4454 generic.go:356] "Generic (PLEG): container finished" podID="96914100-0306-4d16-b413-97d487e4f6ef" containerID="aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4" exitCode=0
Nov 03 22:05:44.744479 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.744336    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" event={"ID":"96914100-0306-4d16-b413-97d487e4f6ef","Type":"ContainerDied","Data":"aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4"}
Nov 03 22:05:44.744479 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.744349    4454 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"
Nov 03 22:05:44.744479 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.744377    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58" event={"ID":"96914100-0306-4d16-b413-97d487e4f6ef","Type":"ContainerDied","Data":"a3b4e784d2803e13ec5d699a800218080ad97538bc025ab235f579bea801b499"}
Nov 03 22:05:44.744479 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.744397    4454 scope.go:122] "RemoveContainer" containerID="aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4"
Nov 03 22:05:44.758165 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.758141    4454 scope.go:122] "RemoveContainer" containerID="b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80"
Nov 03 22:05:44.766810 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.766791    4454 scope.go:122] "RemoveContainer" containerID="aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4"
Nov 03 22:05:44.767335 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:05:44.767310    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4\": container with ID starting with aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4 not found: ID does not exist" containerID="aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4"
Nov 03 22:05:44.767425 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.767342    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4"} err="failed to get container status \"aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4\": rpc error: code = NotFound desc = could not find container \"aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4\": container with ID starting with aea4e086d35d86d7b6aae0c482360acdbfdbb6b3dc5f04efd6c7af7cc50d6dd4 not found: ID does not exist"
Nov 03 22:05:44.767425 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.767360    4454 scope.go:122] "RemoveContainer" containerID="b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80"
Nov 03 22:05:44.767706 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:05:44.767681    4454 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80\": container with ID starting with b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80 not found: ID does not exist" containerID="b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80"
Nov 03 22:05:44.767767 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.767721    4454 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80"} err="failed to get container status \"b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80\": rpc error: code = NotFound desc = could not find container \"b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80\": container with ID starting with b1bfde29f7a6512f3213eb654fa9139590a31f2dc41fcd77b31fe60ff7860a80 not found: ID does not exist"
Nov 03 22:05:44.794783 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.794759    4454 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"]
Nov 03 22:05:44.799973 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:44.799948    4454 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-6a697e-openshift-default-5c9b4c95bf-ltr58"]
Nov 03 22:05:45.229008 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:45.228977    4454 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96914100-0306-4d16-b413-97d487e4f6ef" path="/var/lib/kubelet/pods/96914100-0306-4d16-b413-97d487e4f6ef/volumes"
Nov 03 22:05:49.851220 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:05:49.851144    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:05:52.436983 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:05:52.436898    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:06:07.707938 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:06:07.707891    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:06:22.371718 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:06:22.371678    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:06:36.188425 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:06:36.188152    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-x9hfv"]
Nov 03 22:06:36.361494 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:06:36.361461    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-66ccfd4b68-5hxfz_4ca0d8d5-2b74-488a-949a-69b6d17d9d9a/authorino/0.log"
Nov 03 22:06:39.333014 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:06:39.332977    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:06:46.811434 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:06:46.811396    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-76cf67fc7b-5hk42_54a67527-d872-42d9-9e96-fad468debfe8/manager/0.log"
Nov 03 22:06:49.164804 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:06:49.164771    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-cb9fbb88b-56xzv_b5927138-0e11-4836-b0c9-5201e8e5b6e7/maas-api/0.log"
Nov 03 22:06:52.370294 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:06:52.370256    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:07:02.030492 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:02.030451    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-66ccfd4b68-5hxfz_4ca0d8d5-2b74-488a-949a-69b6d17d9d9a/authorino/0.log"
Nov 03 22:07:07.054654 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:07.054614    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:07:13.152453 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:13.152420    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-x9hfv_58409ead-d6be-41bc-abbd-5df50f5ea4bd/limitador/0.log"
Nov 03 22:07:14.310438 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:14.310405    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-hr9cx_0c280d46-c895-4c9d-a21a-2b46f39c59b5/discovery/0.log"
Nov 03 22:07:14.510088 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:14.510064    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b5554b6f8-p8mxx_4cdb1c04-4f27-44a0-b05a-08cb967ccf13/kube-auth-proxy/0.log"
Nov 03 22:07:22.372388 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:07:22.372349    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:07:25.715118 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:25.715081    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-lq9lw_6174dd8c-e88b-48eb-8ba6-9935eb711810/istio-proxy/0.log"
Nov 03 22:07:26.330787 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:26.330756    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-x4f2g_ea097b76-14e5-48e9-9966-ed40754d66f7/istio-proxy/0.log"
Nov 03 22:07:26.524148 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:26.524117    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-x7pth_e86fb6a8-2f81-4752-92bb-adff8cd0f2cf/istio-proxy/0.log"
Nov 03 22:07:27.300162 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:27.300135    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wgf6d_b6845d56-0ca9-4dd3-a778-9849db797e23/payload-processing/0.log"
Nov 03 22:07:28.488915 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:28.488881    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt_d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0/main/0.log"
Nov 03 22:07:29.073146 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:29.073111    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-mwjk9_4ad5a89a-2a2f-451b-8aae-e336df0ed106/main/0.log"
Nov 03 22:07:29.469776 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:29.469742    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-4kqqw_77c0ac9e-c55d-4503-b2e6-2f29fb6f4454/main/0.log"
Nov 03 22:07:29.845346 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:29.845319    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn_f9a60514-d1ae-4072-b77a-9eb51ff05a08/main/0.log"
Nov 03 22:07:30.754546 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:30.754496    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-66ccfd4b68-5hxfz_4ca0d8d5-2b74-488a-949a-69b6d17d9d9a/authorino/0.log"
Nov 03 22:07:39.406425 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:39.406382    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:07:39.824750 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:39.824664    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-76cf67fc7b-5hk42_54a67527-d872-42d9-9e96-fad468debfe8/manager/0.log"
Nov 03 22:07:41.843503 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:41.843451    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-cb9fbb88b-56xzv_b5927138-0e11-4836-b0c9-5201e8e5b6e7/maas-api/0.log"
Nov 03 22:07:52.371185 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:07:52.371144    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:07:55.013467 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:55.013437    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-66ccfd4b68-5hxfz_4ca0d8d5-2b74-488a-949a-69b6d17d9d9a/authorino/0.log"
Nov 03 22:07:56.093258 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:56.093228    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-x9hfv_58409ead-d6be-41bc-abbd-5df50f5ea4bd/limitador/0.log"
Nov 03 22:07:57.027144 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:57.027113    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-hr9cx_0c280d46-c895-4c9d-a21a-2b46f39c59b5/discovery/0.log"
Nov 03 22:07:57.225496 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:57.225462    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b5554b6f8-p8mxx_4cdb1c04-4f27-44a0-b05a-08cb967ccf13/kube-auth-proxy/0.log"
Nov 03 22:07:58.397022 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:58.396990    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-lq9lw_6174dd8c-e88b-48eb-8ba6-9935eb711810/istio-proxy/0.log"
Nov 03 22:07:58.984201 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:58.984160    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-x4f2g_ea097b76-14e5-48e9-9966-ed40754d66f7/istio-proxy/0.log"
Nov 03 22:07:59.170380 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:59.170346    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-x7pth_e86fb6a8-2f81-4752-92bb-adff8cd0f2cf/istio-proxy/0.log"
Nov 03 22:07:59.966460 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:07:59.966429    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wgf6d_b6845d56-0ca9-4dd3-a778-9849db797e23/payload-processing/0.log"
Nov 03 22:08:00.663132 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:08:00.663099    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-8lzqt_d06ae51f-4910-4aa8-aa9d-91fe3dfb49a0/main/0.log"
Nov 03 22:08:01.232543 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:08:01.232496    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-mwjk9_4ad5a89a-2a2f-451b-8aae-e336df0ed106/main/0.log"
Nov 03 22:08:01.599708 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:08:01.599671    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-4kqqw_77c0ac9e-c55d-4503-b2e6-2f29fb6f4454/main/0.log"
Nov 03 22:08:01.963286 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:08:01.963200    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-wbtkn_f9a60514-d1ae-4072-b77a-9eb51ff05a08/main/0.log"
Nov 03 22:08:22.370850 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:08:22.370806    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:08:39.526451 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:08:39.526412    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:08:52.370793 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:08:52.370754    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:09:22.034737 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:09:22.034700    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:09:22.370566 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:09:22.370531    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:09:29.182635 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:09:29.182594    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wgf6d_b6845d56-0ca9-4dd3-a778-9849db797e23/payload-processing/0.log"
Nov 03 22:09:29.375080 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:09:29.375052    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wgf6d_b6845d56-0ca9-4dd3-a778-9849db797e23/payload-processing/0.log"
Nov 03 22:09:39.642035 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:09:39.641947    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:09:51.446291 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:09:51.446260    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 22:09:51.452601 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:09:51.452579    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6xdtq_fe26d35e-47a4-497a-9fe1-64cc807267e9/ovn-acl-logging/0.log"
Nov 03 22:09:52.373062 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:09:52.373021    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:10:12.715882 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:12.715838    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:10:22.371655 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:10:22.371619    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:10:32.931141 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:32.931104    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:10:33.046725 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:33.046663    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:10:39.781562 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:39.781472    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:10:52.373834 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:10:52.373795    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:10:54.766087 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:54.766057    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wgf6d_b6845d56-0ca9-4dd3-a778-9849db797e23/payload-processing/0.log"
Nov 03 22:10:54.945993 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:54.945955    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wgf6d_b6845d56-0ca9-4dd3-a778-9849db797e23/payload-processing/0.log"
Nov 03 22:10:55.238799 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.238765    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sl2kc/must-gather-qrbrb"]
Nov 03 22:10:55.239249 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.239230    4454 cpu_manager.go:414] "containerMap: removing container" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy"
Nov 03 22:10:55.239369 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.239249    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy"
Nov 03 22:10:55.239369 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.239268    4454 cpu_manager.go:414] "containerMap: removing container" podUID="bed92553-a8b7-42c0-8309-10606ae1d416" containerName="maas-api"
Nov 03 22:10:55.239369 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.239276    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bed92553-a8b7-42c0-8309-10606ae1d416" containerName="maas-api"
Nov 03 22:10:55.239369 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.239287    4454 cpu_manager.go:414] "containerMap: removing container" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy"
Nov 03 22:10:55.239369 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.239294    4454 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy"
Nov 03 22:10:55.239665 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.239378    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy"
Nov 03 22:10:55.239665 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.239392    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="bed92553-a8b7-42c0-8309-10606ae1d416" containerName="maas-api"
Nov 03 22:10:55.239665 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.239411    4454 memory_manager.go:365] "RemoveStaleState removing state" podUID="96914100-0306-4d16-b413-97d487e4f6ef" containerName="istio-proxy"
Nov 03 22:10:55.242676 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.242654    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sl2kc/must-gather-qrbrb"
Nov 03 22:10:55.251332 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.251301    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:10:55.256040 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.256018    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sl2kc\"/\"openshift-service-ca.crt\""
Nov 03 22:10:55.256040 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.256036    4454 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sl2kc\"/\"kube-root-ca.crt\""
Nov 03 22:10:55.267402 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.267381    4454 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sl2kc\"/\"default-dockercfg-8mbwx\""
Nov 03 22:10:55.278380 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.278350    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sl2kc/must-gather-qrbrb"]
Nov 03 22:10:55.354970 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.354935    4454 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log" limit=1024
Nov 03 22:10:55.423289 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.423252    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0d355b40-0d01-4718-8660-7bfebcc6df57-must-gather-output\") pod \"must-gather-qrbrb\" (UID: \"0d355b40-0d01-4718-8660-7bfebcc6df57\") " pod="openshift-must-gather-sl2kc/must-gather-qrbrb"
Nov 03 22:10:55.423472 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.423311    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sn9f\" (UniqueName: \"kubernetes.io/projected/0d355b40-0d01-4718-8660-7bfebcc6df57-kube-api-access-9sn9f\") pod \"must-gather-qrbrb\" (UID: \"0d355b40-0d01-4718-8660-7bfebcc6df57\") " pod="openshift-must-gather-sl2kc/must-gather-qrbrb"
Nov 03 22:10:55.524258 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.524161    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0d355b40-0d01-4718-8660-7bfebcc6df57-must-gather-output\") pod \"must-gather-qrbrb\" (UID: \"0d355b40-0d01-4718-8660-7bfebcc6df57\") " pod="openshift-must-gather-sl2kc/must-gather-qrbrb"
Nov 03 22:10:55.524258 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.524239    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sn9f\" (UniqueName: \"kubernetes.io/projected/0d355b40-0d01-4718-8660-7bfebcc6df57-kube-api-access-9sn9f\") pod \"must-gather-qrbrb\" (UID: \"0d355b40-0d01-4718-8660-7bfebcc6df57\") " pod="openshift-must-gather-sl2kc/must-gather-qrbrb"
Nov 03 22:10:55.524543 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.524521    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0d355b40-0d01-4718-8660-7bfebcc6df57-must-gather-output\") pod \"must-gather-qrbrb\" (UID: \"0d355b40-0d01-4718-8660-7bfebcc6df57\") " pod="openshift-must-gather-sl2kc/must-gather-qrbrb"
Nov 03 22:10:55.542017 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.541987    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sn9f\" (UniqueName: \"kubernetes.io/projected/0d355b40-0d01-4718-8660-7bfebcc6df57-kube-api-access-9sn9f\") pod \"must-gather-qrbrb\" (UID: \"0d355b40-0d01-4718-8660-7bfebcc6df57\") " pod="openshift-must-gather-sl2kc/must-gather-qrbrb"
Nov 03 22:10:55.556029 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.556004    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sl2kc/must-gather-qrbrb"
Nov 03 22:10:55.848592 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.848562    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sl2kc/must-gather-qrbrb"]
Nov 03 22:10:55.854098 ip-10-0-137-212 kubenswrapper[4454]: W1103 22:10:55.854069    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d355b40_0d01_4718_8660_7bfebcc6df57.slice/crio-59c390ca791284371c24a727a2e3c2ba7b9aa73af5ba2365e377cf8d280a5272 WatchSource:0}: Error finding container 59c390ca791284371c24a727a2e3c2ba7b9aa73af5ba2365e377cf8d280a5272: Status 404 returned error can't find the container with id 59c390ca791284371c24a727a2e3c2ba7b9aa73af5ba2365e377cf8d280a5272
Nov 03 22:10:55.856435 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:55.856415    4454 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:10:56.127070 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:56.126982    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sl2kc/must-gather-qrbrb" event={"ID":"0d355b40-0d01-4718-8660-7bfebcc6df57","Type":"ContainerStarted","Data":"59c390ca791284371c24a727a2e3c2ba7b9aa73af5ba2365e377cf8d280a5272"}
Nov 03 22:10:57.134042 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:57.134007    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sl2kc/must-gather-qrbrb" event={"ID":"0d355b40-0d01-4718-8660-7bfebcc6df57","Type":"ContainerStarted","Data":"b57767e59f6e245f8142e65bc2696fb64d8e85d057775730c7e1bea731966648"}
Nov 03 22:10:57.134042 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:57.134051    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sl2kc/must-gather-qrbrb" event={"ID":"0d355b40-0d01-4718-8660-7bfebcc6df57","Type":"ContainerStarted","Data":"0dd91fee7a5e0d0813b8c9e6a9fd782161b3133333850164615438aa8251d711"}
Nov 03 22:10:57.172006 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:57.171949    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-sl2kc/must-gather-qrbrb" podStartSLOduration=1.378035796 podStartE2EDuration="2.171920828s" totalImagesPullingTime="793.885032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:10:57.171062513 +0000 UTC m=+2466.049884803" watchObservedRunningTime="2026-11-03 22:10:57.171920828 +0000 UTC m=+2466.050743112"
Nov 03 22:10:59.744460 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:10:59.744426    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vmr9k_b978b73d-666a-4c25-b959-74d3e14c7bbf/global-pull-secret-syncer/0.log"
Nov 03 22:11:00.510531 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:00.510220    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-79tpv_45263f32-066f-4bdf-84b9-a3ebd1f33f90/konnectivity-agent/0.log"
Nov 03 22:11:01.008531 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:01.007719    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-212.ec2.internal_9caeb2eb053e15a50e91a5dd1ebbc6a1/haproxy/0.log"
Nov 03 22:11:07.828856 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:07.828819    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-66ccfd4b68-5hxfz_4ca0d8d5-2b74-488a-949a-69b6d17d9d9a/authorino/0.log"
Nov 03 22:11:08.412065 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:08.412039    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-x9hfv_58409ead-d6be-41bc-abbd-5df50f5ea4bd/limitador/0.log"
Nov 03 22:11:09.625831 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.625747    4454 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"]
Nov 03 22:11:09.632481 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.632010    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.648877 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.648841    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"]
Nov 03 22:11:09.717768 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.717732    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77896225-11c2-456d-a2e3-88473b48c0da-podres\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.717953 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.717782    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77896225-11c2-456d-a2e3-88473b48c0da-sys\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.717953 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.717879    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77896225-11c2-456d-a2e3-88473b48c0da-lib-modules\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.718081 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.717954    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4r97\" (UniqueName: \"kubernetes.io/projected/77896225-11c2-456d-a2e3-88473b48c0da-kube-api-access-k4r97\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.718081 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.718052    4454 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77896225-11c2-456d-a2e3-88473b48c0da-proc\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.818894 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.818851    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77896225-11c2-456d-a2e3-88473b48c0da-lib-modules\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.819073 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.818941    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4r97\" (UniqueName: \"kubernetes.io/projected/77896225-11c2-456d-a2e3-88473b48c0da-kube-api-access-k4r97\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.819073 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.818986    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77896225-11c2-456d-a2e3-88473b48c0da-proc\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.819073 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.818994    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77896225-11c2-456d-a2e3-88473b48c0da-lib-modules\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.819194 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.819081    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77896225-11c2-456d-a2e3-88473b48c0da-proc\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.819194 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.819105    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77896225-11c2-456d-a2e3-88473b48c0da-podres\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.819194 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.819140    4454 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77896225-11c2-456d-a2e3-88473b48c0da-sys\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.819312 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.819209    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77896225-11c2-456d-a2e3-88473b48c0da-podres\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.819312 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.819239    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77896225-11c2-456d-a2e3-88473b48c0da-sys\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.837026 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.836985    4454 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4r97\" (UniqueName: \"kubernetes.io/projected/77896225-11c2-456d-a2e3-88473b48c0da-kube-api-access-k4r97\") pod \"perf-node-gather-daemonset-dw9w5\" (UID: \"77896225-11c2-456d-a2e3-88473b48c0da\") " pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:09.976320 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:09.976236    4454 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:10.277309 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:10.276077    4454 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"]
Nov 03 22:11:10.283636 ip-10-0-137-212 kubenswrapper[4454]: W1103 22:11:10.283600    4454 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod77896225_11c2_456d_a2e3_88473b48c0da.slice/crio-538005cdd99721dc780dbc4a2571deadbc3a2e3c67379cbb6da09d9cf0835549 WatchSource:0}: Error finding container 538005cdd99721dc780dbc4a2571deadbc3a2e3c67379cbb6da09d9cf0835549: Status 404 returned error can't find the container with id 538005cdd99721dc780dbc4a2571deadbc3a2e3c67379cbb6da09d9cf0835549
Nov 03 22:11:10.290203 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:10.290169    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5" event={"ID":"77896225-11c2-456d-a2e3-88473b48c0da","Type":"ContainerStarted","Data":"538005cdd99721dc780dbc4a2571deadbc3a2e3c67379cbb6da09d9cf0835549"}
Nov 03 22:11:10.736895 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:10.736865    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-jhvtg_c7073cc7-24ae-4b80-8f87-a217e0073f77/kube-state-metrics/0.log"
Nov 03 22:11:10.844633 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:10.844598    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-jhvtg_c7073cc7-24ae-4b80-8f87-a217e0073f77/kube-rbac-proxy-main/0.log"
Nov 03 22:11:10.960636 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:10.960606    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-jhvtg_c7073cc7-24ae-4b80-8f87-a217e0073f77/kube-rbac-proxy-self/0.log"
Nov 03 22:11:11.213638 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:11.213606    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-jt8bk_d8902cc5-b1dd-45e0-bdfb-79d23a38b9c8/monitoring-plugin/0.log"
Nov 03 22:11:11.298451 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:11.298411    4454 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5" event={"ID":"77896225-11c2-456d-a2e3-88473b48c0da","Type":"ContainerStarted","Data":"165c4f36bea513ef65132d03795eb70d29574afc1a5fe081e83b95d963e85d16"}
Nov 03 22:11:11.298898 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:11.298874    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:11.335519 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:11.335425    4454 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5" podStartSLOduration=2.335409848 podStartE2EDuration="2.335409848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:11:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:11:11.334091772 +0000 UTC m=+2480.212914090" watchObservedRunningTime="2026-11-03 22:11:11.335409848 +0000 UTC m=+2480.214232132"
Nov 03 22:11:11.673167 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:11.673135    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-svqc2_5484b61e-4f34-4b64-ae30-3c04a5a2e20c/node-exporter/0.log"
Nov 03 22:11:11.786221 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:11.786197    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-svqc2_5484b61e-4f34-4b64-ae30-3c04a5a2e20c/kube-rbac-proxy/0.log"
Nov 03 22:11:11.906659 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:11.906633    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-svqc2_5484b61e-4f34-4b64-ae30-3c04a5a2e20c/init-textfile/0.log"
Nov 03 22:11:12.375826 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:12.375800    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-6r4h2_f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb/kube-rbac-proxy-main/0.log"
Nov 03 22:11:12.491173 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:12.491146    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-6r4h2_f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb/kube-rbac-proxy-self/0.log"
Nov 03 22:11:12.608028 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:12.607999    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-6r4h2_f9b7a4fe-6a0c-4dec-a06b-6755c6d02efb/openshift-state-metrics/0.log"
Nov 03 22:11:13.776742 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:13.776711    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-pqsfq_8acfaa66-20e7-495e-b38c-727510641a05/prometheus-operator-admission-webhook/0.log"
Nov 03 22:11:13.871808 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:13.871778    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-567f78586f-6cjcs_5e865acd-e59e-44f5-bf0b-5ea07a883a64/telemeter-client/0.log"
Nov 03 22:11:13.984640 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:13.984612    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-567f78586f-6cjcs_5e865acd-e59e-44f5-bf0b-5ea07a883a64/reload/0.log"
Nov 03 22:11:14.099833 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:14.099796    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-567f78586f-6cjcs_5e865acd-e59e-44f5-bf0b-5ea07a883a64/kube-rbac-proxy/0.log"
Nov 03 22:11:17.316732 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:17.316698    4454 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sl2kc/perf-node-gather-daemonset-dw9w5"
Nov 03 22:11:21.887934 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:21.887891    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sz7tz_d09b08fd-64c6-4b9e-ba06-273cbedbd58a/dns/0.log"
Nov 03 22:11:22.002255 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:22.002223    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sz7tz_d09b08fd-64c6-4b9e-ba06-273cbedbd58a/kube-rbac-proxy/0.log"
Nov 03 22:11:22.117305 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:22.117276    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-529hb_0fe02b31-bc0e-4ffc-916d-f6cd60b68a70/dns-node-resolver/0.log"
Nov 03 22:11:22.378342 ip-10-0-137-212 kubenswrapper[4454]: E1103 22:11:22.378283    4454 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-prbdm" containerName="extractor"
Nov 03 22:11:23.723047 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:23.723017    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6ccd5f8759-n8x2h_d7137f08-58ce-4f62-b53f-b5e5c725f003/registry/0.log"
Nov 03 22:11:24.055160 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:24.055082    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wjgz5_3a287e20-e2d0-451a-97ce-b4fddffeb788/node-ca/0.log"
Nov 03 22:11:24.465200 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:24.465168    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-hr9cx_0c280d46-c895-4c9d-a21a-2b46f39c59b5/discovery/0.log"
Nov 03 22:11:24.496006 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:24.495978    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b5554b6f8-p8mxx_4cdb1c04-4f27-44a0-b05a-08cb967ccf13/kube-auth-proxy/0.log"
Nov 03 22:11:24.845323 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:24.845291    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-lq9lw_6174dd8c-e88b-48eb-8ba6-9935eb711810/istio-proxy/0.log"
Nov 03 22:11:25.117730 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:25.117653    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-x4f2g_ea097b76-14e5-48e9-9966-ed40754d66f7/istio-proxy/0.log"
Nov 03 22:11:25.175773 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:25.175739    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-x7pth_e86fb6a8-2f81-4752-92bb-adff8cd0f2cf/istio-proxy/0.log"
Nov 03 22:11:25.399468 ip-10-0-137-212 kubenswrapper[4454]: I1103 22:11:25.399394    4454 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wgf6d_b6845d56-0ca9-4dd3-a778-9849db797e23/payload-processing/0.log"