Nov 04 07:12:44.796065 ip-10-0-142-197 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 04 07:12:44.796077 ip-10-0-142-197 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 04 07:12:44.796083 ip-10-0-142-197 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 04 07:12:44.796311 ip-10-0-142-197 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 04 07:12:56.082354 ip-10-0-142-197 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 04 07:12:56.082372 ip-10-0-142-197 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot aeb749f45bf14fcd814ae26a36c372ca --
Nov 04 07:15:06.064652 ip-10-0-142-197 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 07:15:06.490239 ip-10-0-142-197 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 07:15:06.490239 ip-10-0-142-197 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 07:15:06.490239 ip-10-0-142-197 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 07:15:06.490239 ip-10-0-142-197 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 07:15:06.493352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493273    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:15:06.493352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493353    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493357    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493374    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493377    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493380    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493383    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493386    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493389    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493391    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493394    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493397    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493400    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493403    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493405    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493408    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493410    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493413    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493415    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493418    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493421    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:15:06.493440 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493423    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493425    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493428    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493431    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493433    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493443    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493448    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493459    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493464    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493467    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493470    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493473    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493476    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493479    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493481    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493484    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493486    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493489    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493491    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:15:06.493947 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493494    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493496    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493499    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493501    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493504    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493507    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493509    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493512    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493515    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493524    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493527    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493529    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493532    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493534    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493537    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493540    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493543    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493546    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493548    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:15:06.494419 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493551    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493554    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493556    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493559    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493561    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493564    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493567    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493570    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493572    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493575    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493577    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493580    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493582    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493585    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493589    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493591    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493594    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493596    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493599    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:15:06.494865 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493602    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493604    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493607    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493610    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493618    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493621    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493624    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493626    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493634    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493638    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493641    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493645    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493648    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493651    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493653    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493656    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493658    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493661    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493664    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:15:06.495349 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493666    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493669    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493672    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493674    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493677    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493680    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493682    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493685    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.493687    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494475    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494483    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494485    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494488    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494491    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494494    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494497    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494500    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494502    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494506    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494509    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:15:06.495857 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494512    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494516    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494520    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494524    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494527    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494530    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494533    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494536    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494538    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494541    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494544    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494546    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494549    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494551    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494554    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494556    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494559    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494562    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494564    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:15:06.496352 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494568    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494570    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494573    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494576    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494578    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494581    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494583    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494586    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494588    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494591    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494594    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494596    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494600    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494603    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494605    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494609    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494612    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494615    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494617    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494620    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:15:06.496831 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494622    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494625    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494628    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494630    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494633    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494636    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494638    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494642    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494645    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494649    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494651    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494654    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494656    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494659    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494661    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494664    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494666    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494669    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494672    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:15:06.497325 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494675    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494678    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494680    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494683    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494685    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494688    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494691    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494694    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494697    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494700    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494703    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494705    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494708    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494713    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494715    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494718    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494720    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494723    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494725    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:15:06.497848 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494728    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494731    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494733    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494736    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494738    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494741    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494744    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494746    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494749    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494751    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494754    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494756    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494759    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494761    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494764    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494767    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494770    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.494772    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495259    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:15:06.498301 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495265    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495269    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495274    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495278    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495281    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495284    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495287    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495292    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495295    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495298    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495301    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495303    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495306    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495309    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495311    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495314    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495317    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495319    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495322    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:15:06.498775 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495325    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495327    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495330    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495332    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495335    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495337    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495340    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495342    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495345    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495347    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495350    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495353    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495356    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495373    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495376    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495380    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495383    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495385    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495388    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495391    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:15:06.499298 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495394    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495397    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495400    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495402    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495405    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495408    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495410    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495413    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495415    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495418    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495422    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495426    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495429    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495432    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495435    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495437    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495440    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495443    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495446    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:15:06.499783 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495448    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495451    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495453    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495456    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495459    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495461    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495464    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495467    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495471    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495474    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495476    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495479    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495482    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495485    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495488    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495491    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495493    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495495    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495498    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495501    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:15:06.500237 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495503    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495506    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495508    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495511    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495514    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495516    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495519    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495521    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495524    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495526    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495529    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495531    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495534    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495536    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495539    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495541    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495544    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495546    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495550    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495553    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:15:06.500719 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495556    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495566    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495568    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495571    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495574    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495577    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.495580    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496511    2581 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496520    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496527    2581 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496532    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496536    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496540    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496544    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496548    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496552    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496555    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496558    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496562    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496565    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496568    2581 flags.go:64] FLAG: --cgroup-root=""
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496571    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496574    2581 flags.go:64] FLAG: --client-ca-file=""
Nov 04 07:15:06.501180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496577    2581 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496580    2581 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496584    2581 flags.go:64] FLAG: --cluster-domain=""
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496587    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496590    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496593    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496597    2581 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496601    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496604    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496607    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496610    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496614    2581 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496617    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496621    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496624    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496627    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496631    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496634    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496637    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496641    2581 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496644    2581 flags.go:64] FLAG: --enable-server="true"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496647    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496652    2581 flags.go:64] FLAG: --event-burst="100"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496655    2581 flags.go:64] FLAG: --event-qps="50"
Nov 04 07:15:06.501748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496658    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496661    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496664    2581 flags.go:64] FLAG: --eviction-hard=""
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496669    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496672    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496676    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496679    2581 flags.go:64] FLAG: --eviction-soft=""
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496682    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496685    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496688    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496691    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496694    2581 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496697    2581 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496700    2581 flags.go:64] FLAG: --feature-gates=""
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496704    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496707    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496711    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496714    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496717    2581 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496720    2581 flags.go:64] FLAG: --help="false"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496723    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496729    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496732    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 07:15:06.502391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496735    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496738    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496742    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496745    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496747    2581 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496750    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496753    2581 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496756    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496759    2581 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496762    2581 flags.go:64] FLAG: --kube-reserved=""
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496765    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496768    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496771    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496774    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496777    2581 flags.go:64] FLAG: --lock-file=""
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496779    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496782    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496785    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496791    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496794    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496798    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496801    2581 flags.go:64] FLAG: --logging-format="text"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496804    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496807    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 07:15:06.502965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496810    2581 flags.go:64] FLAG: --manifest-url=""
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496813    2581 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496818    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496821    2581 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496830    2581 flags.go:64] FLAG: --max-pods="110"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496833    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496838    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496841    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496845    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496848    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496851    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496854    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496862    2581 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496865    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496868    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496871    2581 flags.go:64] FLAG: --pod-cidr=""
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496874    2581 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496877    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496880    2581 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496883    2581 flags.go:64] FLAG: --port="10250"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496886    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496889    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-08cefc27ae57d773b"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496892    2581 flags.go:64] FLAG: --qos-reserved=""
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496895    2581 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496898    2581 flags.go:64] FLAG: --register-node="true"
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496901    2581 flags.go:64] FLAG: --register-with-taints=""
Nov 04 07:15:06.503543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496905    2581 flags.go:64] FLAG: --registry-burst="10"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496908    2581 flags.go:64] FLAG: --registry-qps="5"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496911    2581 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496913    2581 flags.go:64] FLAG: --reserved-memory=""
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496918    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496921    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496924    2581 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496926    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496929    2581 flags.go:64] FLAG: --runonce="false"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496932    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496936    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496938    2581 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496942    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496946    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496960    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496964    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496967    2581 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496970    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496973    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496976    2581 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496979    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496982    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496985    2581 flags.go:64] FLAG: --system-cgroups=""
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496988    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496993    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 07:15:06.504160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.496996    2581 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.497000    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.497004    2581 flags.go:64] FLAG: --tls-min-version=""
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.497006    2581 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.497010    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.497013    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.497016    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.497019    2581 flags.go:64] FLAG: --v="2"
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.497024    2581 flags.go:64] FLAG: --version="false"
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.497028    2581 flags.go:64] FLAG: --vmodule=""
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.497032    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.497035    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497171    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497174    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497177    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497180    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497183    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497186    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497188    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497192    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497195    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497198    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:15:06.504766 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497201    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497203    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497206    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497208    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497211    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497214    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497216    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497219    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497222    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497224    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497227    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497230    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497234    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497237    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497240    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497243    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497246    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497250    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497253    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497255    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:15:06.505273 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497258    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497261    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497263    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497266    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497269    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497271    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497274    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497276    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497279    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497283    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497286    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497288    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497291    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497294    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497297    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497299    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497302    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497304    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497307    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:15:06.505812 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497310    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497312    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497315    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497318    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497320    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497323    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497325    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497328    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497330    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497333    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497337    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497340    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497343    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497346    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497348    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497351    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497354    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497356    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497370    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497373    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:15:06.506278 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497376    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497378    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497383    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497386    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497389    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497392    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497395    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497398    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497401    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497404    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497406    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497409    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497412    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497416    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497420    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497423    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497425    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497428    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497431    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:15:06.506796 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497434    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497436    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497439    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497444    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497447    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497449    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497452    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497455    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497457    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497460    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497462    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497465    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497468    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497470    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497473    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497477    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497480    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:15:06.507255 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497483    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.497510    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497547    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497552    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497556    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497559    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497562    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497565    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497568    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497571    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497574    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497576    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497579    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497582    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497585    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497587    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:15:06.507680 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497590    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497593    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497595    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497599    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497603    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497607    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497611    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497613    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497617    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497620    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497622    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497625    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497627    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497630    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497634    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497637    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497640    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497643    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497646    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:15:06.508055 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497649    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497652    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497654    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497674    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497678    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497682    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497684    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497687    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497690    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497692    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497695    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497698    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497700    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497703    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497706    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497708    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497712    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497715    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497717    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497720    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:15:06.508515 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497723    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497725    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497728    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497731    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497734    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497737    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497739    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497743    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497746    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497748    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497751    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497755    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497759    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497762    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497765    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497769    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497771    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497774    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497776    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:15:06.508991 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497779    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497782    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497785    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497787    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497790    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497792    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497795    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497797    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497800    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497803    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497813    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497816    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497819    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497822    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497824    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497827    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497830    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497832    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497835    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:15:06.509596 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497838    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497842    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497845    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497848    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497851    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497854    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497857    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497860    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497862    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497866    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497869    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497871    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497874    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497877    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.497880    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.507327    2581 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.507343    2581 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.507370    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.507375    2581 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507426    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:15:06.510254 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507432    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507436    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507439    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507442    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507445    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507448    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507450    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507453    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507456    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507458    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507461    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507464    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507467    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507469    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507472    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507475    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507477    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507480    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507482    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:15:06.511039 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507485    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507488    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507491    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507493    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507496    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507498    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507501    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507504    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507507    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507511    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507514    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507516    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507519    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507521    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507524    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507527    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507529    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507532    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507535    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507537    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:15:06.511532 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507540    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507544    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507548    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507550    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507553    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507556    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507559    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507561    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507564    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507567    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507569    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507572    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507574    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507577    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507579    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507582    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507584    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507587    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507590    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:15:06.512051 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507593    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507595    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507598    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507601    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507604    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507607    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507609    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507612    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507614    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507617    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507620    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507622    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507625    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507627    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507630    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507632    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507635    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507637    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507640    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507642    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:15:06.512514 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507645    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507648    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507650    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507653    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507656    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507658    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507662    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507666    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507669    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507671    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507674    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507677    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507679    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507682    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507685    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507688    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507691    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507694    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507697    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:15:06.513014 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507700    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507702    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507705    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507708    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507711    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507713    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507716    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507718    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.507746    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507888    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507893    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507896    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507899    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507901    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507904    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507907    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:15:06.513558 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507909    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507912    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507915    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507917    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507920    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507923    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507925    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507928    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507930    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507933    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507935    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507938    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507941    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507943    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507946    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507949    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507951    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507954    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507956    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:15:06.513975 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507959    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507962    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507964    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507967    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507970    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507973    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507975    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507977    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507980    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507982    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507985    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507987    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507990    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507992    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507995    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.507997    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508000    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508002    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508005    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:15:06.514570 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508007    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508010    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508012    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508015    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508017    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508020    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508023    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508025    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508028    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508033    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508037    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508041    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508044    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508046    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508049    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508051    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508054    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508057    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508059    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:15:06.515013 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508063    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508067    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508070    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508073    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508075    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508078    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508080    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508083    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508085    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508088    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508092    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508095    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508098    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508101    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508103    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508106    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508108    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508111    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508113    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508116    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:15:06.515513 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508119    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508121    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508124    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508127    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508130    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508133    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508136    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508138    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508141    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508143    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508146    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508149    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508151    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508154    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508157    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508159    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508162    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508164    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508167    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508169    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:15:06.516048 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508172    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508174    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.508199    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508228    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508233    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508236    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508239    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508242    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508245    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508249    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508251    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508254    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508257    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508259    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508262    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:15:06.516537 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508264    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508267    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508270    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508273    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508275    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508278    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508280    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508283    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508285    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508288    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508290    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508293    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508296    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508298    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508302    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508306    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508309    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508312    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508315    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508318    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:15:06.517050 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508322    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508325    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508327    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508330    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508334    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508338    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508341    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508344    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508347    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508349    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508352    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508355    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508370    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508373    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508379    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508382    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508385    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508387    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508390    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:15:06.517867 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508392    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508395    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508398    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508400    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508403    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508405    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508408    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508410    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508413    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508415    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508418    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508420    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508423    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508427    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508429    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508432    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508435    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508437    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508440    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508443    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:15:06.518316 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508446    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508448    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508451    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508454    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508456    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508460    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508463    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508465    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508468    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508471    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508473    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508476    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508479    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508482    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508485    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508488    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508490    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508492    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508495    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508498    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:15:06.519084 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508500    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508503    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508505    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508508    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508510    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508514    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508517    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508519    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508522    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508525    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508528    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508530    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508533    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508535    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:06.508538    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.508588    2581 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.513162    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.514069    2581 server.go:1014] "Starting client certificate rotation"
Nov 04 07:15:06.519626 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.514173    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 07:15:06.520059 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.514223    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 07:15:06.520059 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.516783    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 07:15:06.520059 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.517914    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 07:15:06.527590 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.527575    2581 log.go:25] "Validated CRI v1 runtime API"
Nov 04 07:15:06.531413 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.531398    2581 log.go:25] "Validated CRI v1 image API"
Nov 04 07:15:06.532294 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.532267    2581 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 07:15:06.536164 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.536143    2581 fs.go:135] Filesystem UUIDs: map[0c430ad2-fe0c-4b88-a9aa-e39b84f9f905:/dev/nvme0n1p4 1ee09380-3608-40f8-b4dd-1325bc031102:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Nov 04 07:15:06.536241 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.536161    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 07:15:06.540630 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.540520    2581 manager.go:217] Machine: {Timestamp:2026-11-04 07:15:06.539320578 +0000 UTC m=+0.350113427 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099843 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28419b3d4ba97a94d78d33038463d0 SystemUUID:ec28419b-3d4b-a97a-94d7-8d33038463d0 BootID:aeb749f4-5bf1-4fcd-814a-e26a36c372ca Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e7:8d:6b:15 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e7:8d:6b:15 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:3b:70:36:ee:21 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 07:15:06.540630 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.540630    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 07:15:06.540754 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.540701    2581 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 07:15:06.543855 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.543821    2581 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 07:15:06.544019 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.543857    2581 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-197.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 07:15:06.544060 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.544024    2581 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 07:15:06.544060 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.544032    2581 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 07:15:06.544113 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.544077    2581 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 07:15:06.544113 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.544097    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 07:15:06.545215 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.545205    2581 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 07:15:06.548772 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.548761    2581 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 07:15:06.548894 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.548885    2581 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 07:15:06.549650 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.549641    2581 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 07:15:06.549685 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.549656    2581 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 07:15:06.549685 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.549666    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 07:15:06.549685 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.549675    2581 kubelet.go:405] "Adding apiserver pod source"
Nov 04 07:15:06.549685 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.549683    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 07:15:06.550246 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.550232    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 07:15:06.553944 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.553881    2581 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 07:15:06.555012 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.554992    2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 07:15:06.558748 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.558734    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 07:15:06.559504 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.559491    2581 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 07:15:06.559553 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.559521    2581 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 07:15:06.559673 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.559663    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 07:15:06.559701 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.559679    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 07:15:06.559701 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.559685    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 07:15:06.559701 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.559692    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 07:15:06.559701 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.559697    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 07:15:06.559799 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.559703    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 07:15:06.560204 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.560195    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 07:15:06.560236 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.560205    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 07:15:06.560236 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.560212    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 07:15:06.560236 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.560218    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 07:15:06.560236 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.560227    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 07:15:06.560236 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.560236    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 07:15:06.561484 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.561468    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 07:15:06.561484 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.561480    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 07:15:06.564567 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.564547    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-197.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 07:15:06.565304 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.565290    2581 server.go:1263] "Started kubelet"
Nov 04 07:15:06.565411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.565356    2581 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 07:15:06.565463 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.565392    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 07:15:06.565496 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.565479    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 07:15:06.566078 ip-10-0-142-197 systemd[1]: Started Kubernetes Kubelet.
Nov 04 07:15:06.567240 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.567223    2581 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 07:15:06.567776 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.567738    2581 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 07:15:06.571582 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.571551    2581 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 07:15:06.573868 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.573842    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 07:15:06.574652 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.574635    2581 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 07:15:06.574777 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.574756    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:06.575312 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.575299    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 07:15:06.575407 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.575301    2581 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 07:15:06.575407 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.575327    2581 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 07:15:06.575500 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.575460    2581 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 07:15:06.575500 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.575470    2581 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 07:15:06.575596 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.575586    2581 factory.go:55] Registering systemd factory
Nov 04 07:15:06.575635 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.575611    2581 factory.go:223] Registration of the systemd container factory successfully
Nov 04 07:15:06.575836 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.575823    2581 factory.go:153] Registering CRI-O factory
Nov 04 07:15:06.575898 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.575840    2581 factory.go:223] Registration of the crio container factory successfully
Nov 04 07:15:06.575898 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.575892    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 07:15:06.575998 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.575912    2581 factory.go:103] Registering Raw factory
Nov 04 07:15:06.575998 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.575925    2581 manager.go:1199] Started watching for new ooms in manager
Nov 04 07:15:06.576800 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.576786    2581 manager.go:319] Starting recovery of all containers
Nov 04 07:15:06.579897 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.579868    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 04 07:15:06.580002 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.579977    2581 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-197.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 07:15:06.580145 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.580096    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-197.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 04 07:15:06.581053 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.580055    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-197.us-east-2.compute.internal.18e4c63a9843816a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-197.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-197.us-east-2.compute.internal,},FirstTimestamp:2026-11-04 07:15:06.565263722 +0000 UTC m=+0.376056573,LastTimestamp:2026-11-04 07:15:06.565263722 +0000 UTC m=+0.376056573,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-197.us-east-2.compute.internal,}"
Nov 04 07:15:06.583524 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.583496    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9rc76"
Nov 04 07:15:06.583871 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.583842    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 04 07:15:06.588617 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.588598    2581 manager.go:324] Recovery completed
Nov 04 07:15:06.594043 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.594027    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 07:15:06.595834 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.595818    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 07:15:06.595911 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.595852    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 07:15:06.595911 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.595867    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 07:15:06.596177 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.596153    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9rc76"
Nov 04 07:15:06.596537 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.596523    2581 cpu_manager.go:226] "Starting" policy="none"
Nov 04 07:15:06.596537 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.596531    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 07:15:06.596537 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.596536    2581 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 07:15:06.596708 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.596555    2581 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 07:15:06.597176 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.597159    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 07:15:06.597258 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.597183    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 07:15:06.597258 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.597199    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 07:15:06.598183 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.598104    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-197.us-east-2.compute.internal.18e4c63a9a15f8b3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-197.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-197.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-197.us-east-2.compute.internal,},FirstTimestamp:2026-11-04 07:15:06.595834035 +0000 UTC m=+0.406626885,LastTimestamp:2026-11-04 07:15:06.595834035 +0000 UTC m=+0.406626885,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-197.us-east-2.compute.internal,}"
Nov 04 07:15:06.599235 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.599222    2581 policy_none.go:50] "Start"
Nov 04 07:15:06.599298 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.599241    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 07:15:06.599298 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.599254    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 07:15:06.601300 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.601286    2581 policy_none.go:44] "Start"
Nov 04 07:15:06.627970 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.627940    2581 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 07:15:06.630292 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.630276    2581 manager.go:341] "Starting Device Plugin manager"
Nov 04 07:15:06.630375 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.630306    2581 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 07:15:06.630375 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.630319    2581 server.go:87] "Starting device plugin registration server"
Nov 04 07:15:06.630633 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.630617    2581 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 07:15:06.630714 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.630633    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 07:15:06.630792 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.630776    2581 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 07:15:06.630887 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.630869    2581 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 07:15:06.630887 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.630883    2581 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 07:15:06.631132 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.631111    2581 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 07:15:06.631203 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.631155    2581 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:06.731571 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.731555    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 07:15:06.733542 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.732391    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 07:15:06.733542 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.732416    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 07:15:06.733542 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.732431    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 07:15:06.733542 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.732456    2581 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.738100 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.738079    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 07:15:06.739224 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.739209    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 07:15:06.739290 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.739228    2581 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 07:15:06.739290 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.739246    2581 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 07:15:06.739290 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.739286    2581 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 04 07:15:06.741065 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.741023    2581 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.741065 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.741044    2581 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-197.us-east-2.compute.internal\": node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:06.743678 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.743664    2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 07:15:06.807307 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.807140    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:06.839873 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.839854    2581 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"]
Nov 04 07:15:06.839927 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.839905    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 07:15:06.844111 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.844097    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 07:15:06.844195 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.844124    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 07:15:06.844195 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.844134    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 07:15:06.846391 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.846379    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 07:15:06.846542 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.846530    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.846576 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.846557    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 07:15:06.847116 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.847100    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 07:15:06.847180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.847131    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 07:15:06.847180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.847149    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 07:15:06.847180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.847151    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 07:15:06.847180 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.847172    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 07:15:06.847291 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.847188    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 07:15:06.849496 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.849480    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.849540 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.849510    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 07:15:06.850266 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.850246    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 07:15:06.850332 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.850278    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 07:15:06.850332 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.850288    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 07:15:06.873859 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.873840    2581 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found" node="ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.876561 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.876545    2581 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found" node="ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.877539 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.877521    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42efe434931e8b4798eec39818614b9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"42efe434931e8b4798eec39818614b9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.877590 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.877546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cec163fb92d8094aef96320b2516e2df-config\") pod \"kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"cec163fb92d8094aef96320b2516e2df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.877590 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.877563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42efe434931e8b4798eec39818614b9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"42efe434931e8b4798eec39818614b9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.907679 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:06.907662    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:06.978470 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.978445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cec163fb92d8094aef96320b2516e2df-config\") pod \"kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"cec163fb92d8094aef96320b2516e2df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.978470 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.978472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42efe434931e8b4798eec39818614b9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"42efe434931e8b4798eec39818614b9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.978631 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.978489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42efe434931e8b4798eec39818614b9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"42efe434931e8b4798eec39818614b9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.978631 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.978514    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42efe434931e8b4798eec39818614b9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"42efe434931e8b4798eec39818614b9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.978631 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.978549    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42efe434931e8b4798eec39818614b9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"42efe434931e8b4798eec39818614b9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:06.978631 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:06.978553    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cec163fb92d8094aef96320b2516e2df-config\") pod \"kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"cec163fb92d8094aef96320b2516e2df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:07.008140 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:07.008089    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:07.108657 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:07.108632    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:07.176932 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:07.176912    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:07.180021 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:07.180005    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:07.209489 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:07.209467    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:07.310018 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:07.309953    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:07.411312 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:07.411285    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:07.513479 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:07.513444    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:07.513937 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:07.513490    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 07:15:07.513937 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:07.513653    2581 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 07:15:07.583887 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:07.583802    2581 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 07:15:07.600536 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:07.600494    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 07:10:07 +0000 UTC" deadline="2028-05-08 03:11:43.579280073 +0000 UTC"
Nov 04 07:15:07.600536 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:07.600529    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13219h56m35.978754018s"
Nov 04 07:15:07.613715 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:07.613570    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:07.648419 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:07.648393    2581 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 07:15:07.714022 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:07.713997    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:07.814745 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:07.814707    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:07.826693 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:07.826664    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42efe434931e8b4798eec39818614b9c.slice/crio-5362822ccb25ab840a2cbfd2e4e3e9913d811b7d2d7acd92265f8e42093b7c4e WatchSource:0}: Error finding container 5362822ccb25ab840a2cbfd2e4e3e9913d811b7d2d7acd92265f8e42093b7c4e: Status 404 returned error can't find the container with id 5362822ccb25ab840a2cbfd2e4e3e9913d811b7d2d7acd92265f8e42093b7c4e
Nov 04 07:15:07.828305 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:07.828274    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcec163fb92d8094aef96320b2516e2df.slice/crio-087e7de43b69fa7fe5b19fa18f72cb7a1d2a6e1e49359abbd50084cdf2ced98c WatchSource:0}: Error finding container 087e7de43b69fa7fe5b19fa18f72cb7a1d2a6e1e49359abbd50084cdf2ced98c: Status 404 returned error can't find the container with id 087e7de43b69fa7fe5b19fa18f72cb7a1d2a6e1e49359abbd50084cdf2ced98c
Nov 04 07:15:07.831765 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:07.831742    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 07:15:07.914853 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:07.914819    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:08.016542 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.016505    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:08.117323 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.117243    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:15:08.135409 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.134142    2581 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 07:15:08.176473 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.176449    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:08.189461 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.189444    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 07:15:08.190531 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.190513    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:15:08.202422 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.202399    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 07:15:08.552691 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.552613    2581 apiserver.go:52] "Watching apiserver"
Nov 04 07:15:08.562320 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.562278    2581 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 07:15:08.562725 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.562700    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-qt2pw","openshift-image-registry/node-ca-2pxh9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal","openshift-multus/multus-dn8mz","openshift-network-diagnostics/network-check-target-smfct","kube-system/global-pull-secret-syncer-dxhns","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m","openshift-cluster-node-tuning-operator/tuned-758f6","openshift-multus/multus-additional-cni-plugins-k7449","openshift-ovn-kubernetes/ovnkube-node-2gcvm","openshift-dns/node-resolver-2zd55","openshift-multus/network-metrics-daemon-kr5j5","kube-system/konnectivity-agent-knhlk","kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"]
Nov 04 07:15:08.565472 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.565449    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.568284 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.567829    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:15:08.572521 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.572500    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 07:15:08.572619 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.572527    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 07:15:08.572673 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.572644    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 07:15:08.572710 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.572692    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 07:15:08.572792 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.572774    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8bggl\""
Nov 04 07:15:08.572859 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.572790    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 07:15:08.572914 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.572877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.573704 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.573680    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tjts4\""
Nov 04 07:15:08.573865 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.573846    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.577372 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.577325    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 07:15:08.577717 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.577699    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tqngx\""
Nov 04 07:15:08.577836 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.577820    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 07:15:08.577910 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.577897    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 07:15:08.578166 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.578150    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 07:15:08.578243 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.578226    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 07:15:08.578243 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.578240    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 07:15:08.578354 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.578295    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 07:15:08.580081 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.579714    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:08.580081 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.579807    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:08.580081 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.580045    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-8kctk\""
Nov 04 07:15:08.582367 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.582348    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:08.582485 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.582456    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:08.585246 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.585229    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.588563 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.588545    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:15:08.588983 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.588966    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-conf-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.589064 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.588992    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.589064 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-sys-fs\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.589064 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589024    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-modprobe-d\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.589064 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589041    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysconfig\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.589275 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589065    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-netns\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.589275 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysctl-d\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.589275 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-k8s-cni-cncf-io\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.589275 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48kkb\" (UniqueName: \"kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb\") pod \"network-check-target-smfct\" (UID: \"ab0e4439-974e-420d-be04-e9e86c18d606\") " pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:08.589510 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589276    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-socket-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.589510 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589357    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38-konnectivity-ca\") pod \"konnectivity-agent-knhlk\" (UID: \"13da5ce6-d81c-49c2-b92b-ffb77d6a2e38\") " pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:15:08.589510 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589409    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-lib-modules\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.589510 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589435    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03300026-6052-474a-a6b0-53afd4f3f67a-etc-tuned\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.589510 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589458    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-cnibin\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.589510 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-socket-dir-parent\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.589510 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589509    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpmn2\" (UniqueName: \"kubernetes.io/projected/caee6966-5544-44a7-8b90-c431ef6b94de-kube-api-access-bpmn2\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.589875 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589534    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-kubernetes\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.589875 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589558    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-run\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.589875 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.589875 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589662    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-var-lib-kubelet\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.589875 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-cni-bin\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.589875 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-cni-multus\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.589875 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589769    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysctl-conf\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.589875 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-systemd\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.589875 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-system-cni-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.589875 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589886    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-cni-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/caee6966-5544-44a7-8b90-c431ef6b94de-cni-binary-copy\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-multus-certs\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.589972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59cl7\" (UniqueName: \"kubernetes.io/projected/07aa084c-c07a-4901-800d-a3ba2a659bd5-kube-api-access-59cl7\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.590002    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-sys\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.590030    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-kubelet\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.590056    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-hostroot\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.590122    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/caee6966-5544-44a7-8b90-c431ef6b94de-multus-daemon-config\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.590151    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-etc-kubernetes\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.590211    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-registration-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.590241    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-device-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.590270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38-agent-certs\") pod \"konnectivity-agent-knhlk\" (UID: \"13da5ce6-d81c-49c2-b92b-ffb77d6a2e38\") " pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.590296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-host\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.590321    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03300026-6052-474a-a6b0-53afd4f3f67a-tmp\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.590397 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.590350    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4qxf\" (UniqueName: \"kubernetes.io/projected/03300026-6052-474a-a6b0-53afd4f3f67a-kube-api-access-n4qxf\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.591075 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.590520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:15:08.591679 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.591654    2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 07:15:08.594578 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.593792    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.601108 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.600661    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:15:08.603325 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.603298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:08.603433 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.603406    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:08.616922 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.616896    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rpvw7"
Nov 04 07:15:08.630257 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.629687    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rpvw7"
Nov 04 07:15:08.672764 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.672731    2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 07:15:08.677013 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.676993    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 07:15:08.678873 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.678839    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 07:15:08.679108 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.679092    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 07:15:08.679188 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.679177    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 07:15:08.679346 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.679331    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 07:15:08.679411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.679387    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 07:15:08.679483 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.679474    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 07:15:08.680279 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.680257    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ltv7r\""
Nov 04 07:15:08.680384 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.680348    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 07:15:08.680473 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.680457    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7qf26\""
Nov 04 07:15:08.680533 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.680515    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l4l9x\""
Nov 04 07:15:08.680585 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.680549    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 07:15:08.680585 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.680573    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 07:15:08.680728 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.680713    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-slv86\""
Nov 04 07:15:08.680790 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.680774    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 07:15:08.680790 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.680786    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 07:15:08.680890 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.680819    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 07:15:08.680890 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.680833    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 07:15:08.680890 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.680844    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 07:15:08.682198 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.682130    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 07:15:08.682198 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.682190    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 07:15:08.682333 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.682237    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gcxz6\""
Nov 04 07:15:08.690848 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.690826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-env-overrides\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.690945 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.690866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-etc-kubernetes\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.690945 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.690894    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-ovn\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.690945 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.690920    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-ovnkube-config\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.691105 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691000    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-etc-kubernetes\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.691105 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691045    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f95c6e6a-6787-423e-8129-a62feb0a5156-hosts-file\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:15:08.691105 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691076    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-device-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.691262 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691121    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38-agent-certs\") pod \"konnectivity-agent-knhlk\" (UID: \"13da5ce6-d81c-49c2-b92b-ffb77d6a2e38\") " pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:15:08.691262 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691182    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-device-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.691262 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03300026-6052-474a-a6b0-53afd4f3f67a-tmp\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.691416 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691284    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4qxf\" (UniqueName: \"kubernetes.io/projected/03300026-6052-474a-a6b0-53afd4f3f67a-kube-api-access-n4qxf\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.691416 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691311    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-conf-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.691416 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcd2d\" (UniqueName: \"kubernetes.io/projected/f95c6e6a-6787-423e-8129-a62feb0a5156-kube-api-access-vcd2d\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:15:08.691593 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691497    2581 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 07:15:08.691795 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691775    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-conf-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.692009 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cnibin\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.692009 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.691996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.692135 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-sys-fs\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.692135 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692066    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysconfig\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.692135 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692088    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-netns\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.692135 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692113    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-etc-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.692135 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692120    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysconfig\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.692135 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692128    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-sys-fs\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.692459 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-ovnkube-script-lib\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.692459 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692086    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.692459 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692172    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-iptables-alerter-script\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:15:08.692459 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692207    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-netns\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.692459 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48kkb\" (UniqueName: \"kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb\") pod \"network-check-target-smfct\" (UID: \"ab0e4439-974e-420d-be04-e9e86c18d606\") " pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:08.692459 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692285    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:08.692459 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692355    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38-konnectivity-ca\") pod \"konnectivity-agent-knhlk\" (UID: \"13da5ce6-d81c-49c2-b92b-ffb77d6a2e38\") " pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:15:08.692459 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692432    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd74j\" (UniqueName: \"kubernetes.io/projected/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-kube-api-access-xd74j\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:15:08.692869 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tvmk\" (UniqueName: \"kubernetes.io/projected/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-kube-api-access-6tvmk\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.692869 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.692869 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692596    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-cni-netd\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.692869 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692640    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-kubelet-config\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:08.692869 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.692869 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-cni-bin\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.692869 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-cni-multus\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.692869 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692803    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-cni-bin\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.692869 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysctl-conf\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.692869 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692864    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-systemd\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692888    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-system-cni-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-cni-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692937    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/caee6966-5544-44a7-8b90-c431ef6b94de-cni-binary-copy\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-multus-certs\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692957    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38-konnectivity-ca\") pod \"konnectivity-agent-knhlk\" (UID: \"13da5ce6-d81c-49c2-b92b-ffb77d6a2e38\") " pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.692993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-slash\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693054    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-dbus\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59cl7\" (UniqueName: \"kubernetes.io/projected/07aa084c-c07a-4901-800d-a3ba2a659bd5-kube-api-access-59cl7\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693110    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-sys\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693135    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-kubelet\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693162    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-hostroot\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/caee6966-5544-44a7-8b90-c431ef6b94de-multus-daemon-config\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693232    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-var-lib-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-system-cni-dir\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693287    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-host-slash\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-registration-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.693370 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-host\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-systemd\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693514    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-host\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693544    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/caee6966-5544-44a7-8b90-c431ef6b94de-cni-binary-copy\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693542    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zrpx\" (UniqueName: \"kubernetes.io/projected/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-kube-api-access-5zrpx\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-modprobe-d\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693602    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-multus-certs\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-log-socket\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysctl-d\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693749    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-cni-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-k8s-cni-cncf-io\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693780    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-run-netns\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693789    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-kubelet\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693806    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rt4f\" (UniqueName: \"kubernetes.io/projected/542b9c39-a718-4064-9640-5123b2dcb358-kube-api-access-7rt4f\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693832    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h7nn\" (UniqueName: \"kubernetes.io/projected/7a10a31c-6c0e-45e1-9d6f-e1532180e474-kube-api-access-8h7nn\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:08.694267 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cni-binary-copy\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693887    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-socket-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693930    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysctl-conf\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.693831    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-host\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-lib-modules\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03300026-6052-474a-a6b0-53afd4f3f67a-etc-tuned\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694056    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysctl-d\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694065    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-cnibin\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694092    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-socket-dir-parent\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694097    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-k8s-cni-cncf-io\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694134    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-cni-multus\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694183    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-socket-dir-parent\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694199    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-socket-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694235    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-hostroot\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694235    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-system-cni-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694303    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694348    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-systemd\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694527    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-sys\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.695131 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694587    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpmn2\" (UniqueName: \"kubernetes.io/projected/caee6966-5544-44a7-8b90-c431ef6b94de-kube-api-access-bpmn2\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694591    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-registration-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694643    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-cni-bin\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694673    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f95c6e6a-6787-423e-8129-a62feb0a5156-tmp-dir\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694674    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/caee6966-5544-44a7-8b90-c431ef6b94de-multus-daemon-config\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694732    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-modprobe-d\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-kubernetes\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-run\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694829    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-systemd-units\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694853    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-cnibin\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694884    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-lib-modules\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694897    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694923    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-node-log\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694935    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-run\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-serviceca\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.694987    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-kubernetes\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.698191 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.695033    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.699616 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.695164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-var-lib-kubelet\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.699616 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.695222    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-var-lib-kubelet\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.699616 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.695262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/542b9c39-a718-4064-9640-5123b2dcb358-ovn-node-metrics-cert\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.699616 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.695297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.699616 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.695348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-kubelet\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.699616 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.696066    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03300026-6052-474a-a6b0-53afd4f3f67a-tmp\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.699616 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.698498    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03300026-6052-474a-a6b0-53afd4f3f67a-etc-tuned\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.702003 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.701899    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38-agent-certs\") pod \"konnectivity-agent-knhlk\" (UID: \"13da5ce6-d81c-49c2-b92b-ffb77d6a2e38\") " pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:15:08.703208 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.703000    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 07:15:08.703208 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.703031    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 07:15:08.703208 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.703045    2581 projected.go:196] Error preparing data for projected volume kube-api-access-48kkb for pod openshift-network-diagnostics/network-check-target-smfct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:08.703208 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.703109    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb podName:ab0e4439-974e-420d-be04-e9e86c18d606 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:09.203086396 +0000 UTC m=+3.013879249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-48kkb" (UniqueName: "kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb") pod "network-check-target-smfct" (UID: "ab0e4439-974e-420d-be04-e9e86c18d606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:08.705836 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.705653    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4qxf\" (UniqueName: \"kubernetes.io/projected/03300026-6052-474a-a6b0-53afd4f3f67a-kube-api-access-n4qxf\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.715956 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.715932    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpmn2\" (UniqueName: \"kubernetes.io/projected/caee6966-5544-44a7-8b90-c431ef6b94de-kube-api-access-bpmn2\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.718090 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.718066    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59cl7\" (UniqueName: \"kubernetes.io/projected/07aa084c-c07a-4901-800d-a3ba2a659bd5-kube-api-access-59cl7\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.746654 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.746606    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal" event={"ID":"cec163fb92d8094aef96320b2516e2df","Type":"ContainerStarted","Data":"087e7de43b69fa7fe5b19fa18f72cb7a1d2a6e1e49359abbd50084cdf2ced98c"}
Nov 04 07:15:08.750383 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.750355    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal" event={"ID":"42efe434931e8b4798eec39818614b9c","Type":"ContainerStarted","Data":"5362822ccb25ab840a2cbfd2e4e3e9913d811b7d2d7acd92265f8e42093b7c4e"}
Nov 04 07:15:08.796603 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-systemd\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.796747 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-host\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:15:08.796747 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zrpx\" (UniqueName: \"kubernetes.io/projected/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-kube-api-access-5zrpx\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:15:08.796747 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-log-socket\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.796747 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796704    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.796747 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796732    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.796978 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-run-netns\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.796978 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rt4f\" (UniqueName: \"kubernetes.io/projected/542b9c39-a718-4064-9640-5123b2dcb358-kube-api-access-7rt4f\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.796978 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h7nn\" (UniqueName: \"kubernetes.io/projected/7a10a31c-6c0e-45e1-9d6f-e1532180e474-kube-api-access-8h7nn\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:08.796978 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cni-binary-copy\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.796978 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796864    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-cni-bin\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.796978 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796890    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f95c6e6a-6787-423e-8129-a62feb0a5156-tmp-dir\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:15:08.796978 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-systemd-units\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.796978 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796940    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.796978 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-node-log\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.796989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-serviceca\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797016    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/542b9c39-a718-4064-9640-5123b2dcb358-ovn-node-metrics-cert\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797095    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797121    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-kubelet\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-env-overrides\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797172    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-ovn\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-ovnkube-config\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f95c6e6a-6787-423e-8129-a62feb0a5156-hosts-file\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcd2d\" (UniqueName: \"kubernetes.io/projected/f95c6e6a-6787-423e-8129-a62feb0a5156-kube-api-access-vcd2d\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cnibin\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797310    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-etc-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797344    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-ovnkube-script-lib\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.797411 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-iptables-alerter-script\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd74j\" (UniqueName: \"kubernetes.io/projected/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-kube-api-access-xd74j\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tvmk\" (UniqueName: \"kubernetes.io/projected/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-kube-api-access-6tvmk\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797551    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797579    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-cni-netd\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-kubelet-config\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-slash\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-dbus\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-var-lib-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-system-cni-dir\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797754    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-host-slash\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797845    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-host-slash\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797895    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-systemd\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.798187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.797941    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-host\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:15:08.798959 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.798202    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-log-socket\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.798959 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.798263    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.798959 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.798862    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.798959 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.798934    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-run-netns\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.799296 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799252    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f95c6e6a-6787-423e-8129-a62feb0a5156-hosts-file\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:15:08.799434 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799358    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.799434 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799359    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-cni-bin\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.799434 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799424    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-cni-netd\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.799598 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799500    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-kubelet-config\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:08.799598 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799501    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cnibin\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.799598 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799563    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-etc-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.799769 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799644    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f95c6e6a-6787-423e-8129-a62feb0a5156-tmp-dir\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:15:08.799769 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799694    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-systemd-units\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.799769 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799704    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-slash\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.799769 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799727    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.799769 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799757    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-node-log\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.800029 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799780    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-dbus\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:08.800029 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799826    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-var-lib-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.800029 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.799870    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-system-cni-dir\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.800200 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.800154    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-serviceca\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:15:08.800200 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.800158    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-ovnkube-script-lib\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.800279 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.800262    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:08.800340 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.800329    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret podName:bd7ea631-0ca9-4829-8408-8352e1d1bc7f nodeName:}" failed. No retries permitted until 2026-11-04 07:15:09.300310787 +0000 UTC m=+3.111103647 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret") pod "global-pull-secret-syncer-dxhns" (UID: "bd7ea631-0ca9-4829-8408-8352e1d1bc7f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:08.800499 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.800387    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cni-binary-copy\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.800574 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.800549    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:08.800645 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:08.800600    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs podName:7a10a31c-6c0e-45e1-9d6f-e1532180e474 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:09.300585444 +0000 UTC m=+3.111378299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs") pod "network-metrics-daemon-kr5j5" (UID: "7a10a31c-6c0e-45e1-9d6f-e1532180e474") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:08.800706 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.800653    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-ovn\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.800826 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.800803    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-env-overrides\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.800890 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.800875    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-kubelet\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.800943 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.800908    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.801264 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.801237    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.801534 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.801482    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-iptables-alerter-script\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:15:08.801635 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.801571    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-ovnkube-config\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.802994 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.802942    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/542b9c39-a718-4064-9640-5123b2dcb358-ovn-node-metrics-cert\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.811664 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.811638    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zrpx\" (UniqueName: \"kubernetes.io/projected/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-kube-api-access-5zrpx\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:15:08.816401 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.816320    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcd2d\" (UniqueName: \"kubernetes.io/projected/f95c6e6a-6787-423e-8129-a62feb0a5156-kube-api-access-vcd2d\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:15:08.817657 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.817629    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h7nn\" (UniqueName: \"kubernetes.io/projected/7a10a31c-6c0e-45e1-9d6f-e1532180e474-kube-api-access-8h7nn\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:08.817951 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.817926    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tvmk\" (UniqueName: \"kubernetes.io/projected/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-kube-api-access-6tvmk\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.818832 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.818808    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd74j\" (UniqueName: \"kubernetes.io/projected/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-kube-api-access-xd74j\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:15:08.819287 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.819262    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rt4f\" (UniqueName: \"kubernetes.io/projected/542b9c39-a718-4064-9640-5123b2dcb358-kube-api-access-7rt4f\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.892009 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.891972    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:15:08.898854 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.898827    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:15:08.904716 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:08.904685    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07aa084c_c07a_4901_800d_a3ba2a659bd5.slice/crio-3f8e8acf59d0d1e1415ce37b581e593e8ad65763b87c6f7529bd85ce14edbe7e WatchSource:0}: Error finding container 3f8e8acf59d0d1e1415ce37b581e593e8ad65763b87c6f7529bd85ce14edbe7e: Status 404 returned error can't find the container with id 3f8e8acf59d0d1e1415ce37b581e593e8ad65763b87c6f7529bd85ce14edbe7e
Nov 04 07:15:08.908996 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:08.908970    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13da5ce6_d81c_49c2_b92b_ffb77d6a2e38.slice/crio-dbd3e16554021dab0d69cbf5ae8293127e610701a980ff0de8eecf00e2a7cb89 WatchSource:0}: Error finding container dbd3e16554021dab0d69cbf5ae8293127e610701a980ff0de8eecf00e2a7cb89: Status 404 returned error can't find the container with id dbd3e16554021dab0d69cbf5ae8293127e610701a980ff0de8eecf00e2a7cb89
Nov 04 07:15:08.909777 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.909755    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:15:08.918539 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.918517    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dn8mz"
Nov 04 07:15:08.919870 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:08.919849    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03300026_6052_474a_a6b0_53afd4f3f67a.slice/crio-6bbe80ac6af21ea95fbf46ec6145c0a4d6437be18a4ac7ecf7216c623a22a629 WatchSource:0}: Error finding container 6bbe80ac6af21ea95fbf46ec6145c0a4d6437be18a4ac7ecf7216c623a22a629: Status 404 returned error can't find the container with id 6bbe80ac6af21ea95fbf46ec6145c0a4d6437be18a4ac7ecf7216c623a22a629
Nov 04 07:15:08.929946 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:08.929919    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaee6966_5544_44a7_8b90_c431ef6b94de.slice/crio-ebeed33edf2dbe9ce87c4318b7629823a1c51d90cdc7bcfca50af8354d9235a1 WatchSource:0}: Error finding container ebeed33edf2dbe9ce87c4318b7629823a1c51d90cdc7bcfca50af8354d9235a1: Status 404 returned error can't find the container with id ebeed33edf2dbe9ce87c4318b7629823a1c51d90cdc7bcfca50af8354d9235a1
Nov 04 07:15:08.942260 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.942236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:15:08.944549 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.944524    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:15:08.947524 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.947506    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:15:08.954782 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:08.954758    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d74bcca_3fbc_4984_8acf_b53b0f34e30a.slice/crio-56db6e6952766514314b8f6471698a3b60340778e554eda21c9809ab4edb5557 WatchSource:0}: Error finding container 56db6e6952766514314b8f6471698a3b60340778e554eda21c9809ab4edb5557: Status 404 returned error can't find the container with id 56db6e6952766514314b8f6471698a3b60340778e554eda21c9809ab4edb5557
Nov 04 07:15:08.962846 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:08.962822    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9ab2f4_8f74_484b_b0ed_ac13ff8e26c6.slice/crio-c1577594e6a15cedeff5be6a89d191f37b2b9095f9c92a1ddc7081cbf6b144ba WatchSource:0}: Error finding container c1577594e6a15cedeff5be6a89d191f37b2b9095f9c92a1ddc7081cbf6b144ba: Status 404 returned error can't find the container with id c1577594e6a15cedeff5be6a89d191f37b2b9095f9c92a1ddc7081cbf6b144ba
Nov 04 07:15:08.964703 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:08.964677    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f1d3530_4c03_4633_92b1_9fc2ab954b9f.slice/crio-dbdf490e980d741d51b0cbaf6cd1551606971f6c843afd7fb336e9eeaef7be98 WatchSource:0}: Error finding container dbdf490e980d741d51b0cbaf6cd1551606971f6c843afd7fb336e9eeaef7be98: Status 404 returned error can't find the container with id dbdf490e980d741d51b0cbaf6cd1551606971f6c843afd7fb336e9eeaef7be98
Nov 04 07:15:08.967973 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.967951    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:15:08.973282 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:08.973265    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:08.980493 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:08.980465    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95c6e6a_6787_423e_8129_a62feb0a5156.slice/crio-e2ce534a29576f7fe285a3207595ab17683ee456d4584eb52d3abd8d7e87e374 WatchSource:0}: Error finding container e2ce534a29576f7fe285a3207595ab17683ee456d4584eb52d3abd8d7e87e374: Status 404 returned error can't find the container with id e2ce534a29576f7fe285a3207595ab17683ee456d4584eb52d3abd8d7e87e374
Nov 04 07:15:09.306514 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.303342    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:09.306514 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.303445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48kkb\" (UniqueName: \"kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb\") pod \"network-check-target-smfct\" (UID: \"ab0e4439-974e-420d-be04-e9e86c18d606\") " pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:09.306514 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.303486    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:09.306793 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:09.306572    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:09.306793 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:09.306644    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:09.306793 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:09.306667    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret podName:bd7ea631-0ca9-4829-8408-8352e1d1bc7f nodeName:}" failed. No retries permitted until 2026-11-04 07:15:10.306646029 +0000 UTC m=+4.117438883 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret") pod "global-pull-secret-syncer-dxhns" (UID: "bd7ea631-0ca9-4829-8408-8352e1d1bc7f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:09.306793 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:09.306706    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs podName:7a10a31c-6c0e-45e1-9d6f-e1532180e474 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:10.306693332 +0000 UTC m=+4.117486171 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs") pod "network-metrics-daemon-kr5j5" (UID: "7a10a31c-6c0e-45e1-9d6f-e1532180e474") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:09.306793 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:09.306725    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 07:15:09.306793 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:09.306749    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 07:15:09.306793 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:09.306772    2581 projected.go:196] Error preparing data for projected volume kube-api-access-48kkb for pod openshift-network-diagnostics/network-check-target-smfct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:09.307152 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:09.306829    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb podName:ab0e4439-974e-420d-be04-e9e86c18d606 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:10.306807175 +0000 UTC m=+4.117600016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-48kkb" (UniqueName: "kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb") pod "network-check-target-smfct" (UID: "ab0e4439-974e-420d-be04-e9e86c18d606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:09.632757 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.632571    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 07:10:08 +0000 UTC" deadline="2028-04-09 05:59:44.313805866 +0000 UTC"
Nov 04 07:15:09.632757 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.632640    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12526h44m34.681170627s"
Nov 04 07:15:09.760135 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.760094    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pxh9" event={"ID":"3f1d3530-4c03-4633-92b1-9fc2ab954b9f","Type":"ContainerStarted","Data":"dbdf490e980d741d51b0cbaf6cd1551606971f6c843afd7fb336e9eeaef7be98"}
Nov 04 07:15:09.765697 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.765664    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k7449" event={"ID":"0d74bcca-3fbc-4984-8acf-b53b0f34e30a","Type":"ContainerStarted","Data":"56db6e6952766514314b8f6471698a3b60340778e554eda21c9809ab4edb5557"}
Nov 04 07:15:09.772034 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.771992    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dn8mz" event={"ID":"caee6966-5544-44a7-8b90-c431ef6b94de","Type":"ContainerStarted","Data":"ebeed33edf2dbe9ce87c4318b7629823a1c51d90cdc7bcfca50af8354d9235a1"}
Nov 04 07:15:09.789865 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.789833    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-knhlk" event={"ID":"13da5ce6-d81c-49c2-b92b-ffb77d6a2e38","Type":"ContainerStarted","Data":"dbd3e16554021dab0d69cbf5ae8293127e610701a980ff0de8eecf00e2a7cb89"}
Nov 04 07:15:09.807631 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.807593    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m" event={"ID":"07aa084c-c07a-4901-800d-a3ba2a659bd5","Type":"ContainerStarted","Data":"3f8e8acf59d0d1e1415ce37b581e593e8ad65763b87c6f7529bd85ce14edbe7e"}
Nov 04 07:15:09.816167 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.816140    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm" event={"ID":"542b9c39-a718-4064-9640-5123b2dcb358","Type":"ContainerStarted","Data":"750a8d71e0a350c6489ceec8d75f40a7e652c9ebeca3a5ddb74a05a355aefad3"}
Nov 04 07:15:09.820371 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.820338    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2zd55" event={"ID":"f95c6e6a-6787-423e-8129-a62feb0a5156","Type":"ContainerStarted","Data":"e2ce534a29576f7fe285a3207595ab17683ee456d4584eb52d3abd8d7e87e374"}
Nov 04 07:15:09.830563 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.830531    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qt2pw" event={"ID":"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6","Type":"ContainerStarted","Data":"c1577594e6a15cedeff5be6a89d191f37b2b9095f9c92a1ddc7081cbf6b144ba"}
Nov 04 07:15:09.842746 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:09.842671    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-758f6" event={"ID":"03300026-6052-474a-a6b0-53afd4f3f67a","Type":"ContainerStarted","Data":"6bbe80ac6af21ea95fbf46ec6145c0a4d6437be18a4ac7ecf7216c623a22a629"}
Nov 04 07:15:10.313428 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:10.312456    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48kkb\" (UniqueName: \"kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb\") pod \"network-check-target-smfct\" (UID: \"ab0e4439-974e-420d-be04-e9e86c18d606\") " pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:10.313428 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:10.312520    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:10.313428 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:10.312585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:10.313428 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:10.312691    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:10.313428 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:10.312745    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret podName:bd7ea631-0ca9-4829-8408-8352e1d1bc7f nodeName:}" failed. No retries permitted until 2026-11-04 07:15:12.312726853 +0000 UTC m=+6.123519693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret") pod "global-pull-secret-syncer-dxhns" (UID: "bd7ea631-0ca9-4829-8408-8352e1d1bc7f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:10.313428 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:10.313191    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 07:15:10.313428 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:10.313212    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 07:15:10.313428 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:10.313226    2581 projected.go:196] Error preparing data for projected volume kube-api-access-48kkb for pod openshift-network-diagnostics/network-check-target-smfct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:10.313428 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:10.313278    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb podName:ab0e4439-974e-420d-be04-e9e86c18d606 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:12.313262103 +0000 UTC m=+6.124054948 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-48kkb" (UniqueName: "kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb") pod "network-check-target-smfct" (UID: "ab0e4439-974e-420d-be04-e9e86c18d606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:10.313428 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:10.313353    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:10.313428 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:10.313393    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs podName:7a10a31c-6c0e-45e1-9d6f-e1532180e474 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:12.313379736 +0000 UTC m=+6.124172585 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs") pod "network-metrics-daemon-kr5j5" (UID: "7a10a31c-6c0e-45e1-9d6f-e1532180e474") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:10.633946 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:10.633854    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 07:10:08 +0000 UTC" deadline="2028-07-30 10:03:42.395482136 +0000 UTC"
Nov 04 07:15:10.633946 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:10.633895    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15218h48m31.761592508s"
Nov 04 07:15:10.745815 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:10.740553    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:10.745815 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:10.740700    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:10.745815 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:10.741170    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:10.745815 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:10.741295    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:10.745815 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:10.741679    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:10.745815 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:10.741786    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:10.858661 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:10.858624    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal" event={"ID":"cec163fb92d8094aef96320b2516e2df","Type":"ContainerStarted","Data":"a3a0be308b7114969659f09b9c9d4eaacbe522e506a105c11bfa37a9b4452040"}
Nov 04 07:15:10.864335 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:10.863744    2581 generic.go:356] "Generic (PLEG): container finished" podID="42efe434931e8b4798eec39818614b9c" containerID="ca176a6ef40dde0757cf8902da6384c12e387b011314867ad762d82e0f3b8c53" exitCode=0
Nov 04 07:15:10.864335 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:10.863793    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal" event={"ID":"42efe434931e8b4798eec39818614b9c","Type":"ContainerDied","Data":"ca176a6ef40dde0757cf8902da6384c12e387b011314867ad762d82e0f3b8c53"}
Nov 04 07:15:10.901875 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:10.901644    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal" podStartSLOduration=2.90162633 podStartE2EDuration="2.90162633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:10.88108551 +0000 UTC m=+4.691878377" watchObservedRunningTime="2026-11-04 07:15:10.90162633 +0000 UTC m=+4.712419194"
Nov 04 07:15:11.879975 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:11.879817    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal" event={"ID":"42efe434931e8b4798eec39818614b9c","Type":"ContainerStarted","Data":"590331da51ad2be7afca506215625db73809bc52cc293d967fa2c1d9281d844b"}
Nov 04 07:15:11.899309 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:11.898576    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal" podStartSLOduration=3.898554575 podStartE2EDuration="3.898554575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:11.897893498 +0000 UTC m=+5.708686363" watchObservedRunningTime="2026-11-04 07:15:11.898554575 +0000 UTC m=+5.709347438"
Nov 04 07:15:12.331759 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:12.330800    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:12.331759 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:12.330870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48kkb\" (UniqueName: \"kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb\") pod \"network-check-target-smfct\" (UID: \"ab0e4439-974e-420d-be04-e9e86c18d606\") " pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:12.331759 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:12.330898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:12.331759 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:12.331024    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:12.331759 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:12.331091    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs podName:7a10a31c-6c0e-45e1-9d6f-e1532180e474 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:16.331070316 +0000 UTC m=+10.141863170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs") pod "network-metrics-daemon-kr5j5" (UID: "7a10a31c-6c0e-45e1-9d6f-e1532180e474") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:12.331759 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:12.331518    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:12.331759 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:12.331565    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret podName:bd7ea631-0ca9-4829-8408-8352e1d1bc7f nodeName:}" failed. No retries permitted until 2026-11-04 07:15:16.331549367 +0000 UTC m=+10.142342220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret") pod "global-pull-secret-syncer-dxhns" (UID: "bd7ea631-0ca9-4829-8408-8352e1d1bc7f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:12.331759 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:12.331650    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 07:15:12.331759 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:12.331664    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 07:15:12.331759 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:12.331677    2581 projected.go:196] Error preparing data for projected volume kube-api-access-48kkb for pod openshift-network-diagnostics/network-check-target-smfct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:12.331759 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:12.331716    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb podName:ab0e4439-974e-420d-be04-e9e86c18d606 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:16.33170362 +0000 UTC m=+10.142496462 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-48kkb" (UniqueName: "kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb") pod "network-check-target-smfct" (UID: "ab0e4439-974e-420d-be04-e9e86c18d606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:12.743220 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:12.743179    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:12.743418 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:12.743327    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:12.743799 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:12.743775    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:12.743908 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:12.743888    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:12.743972 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:12.743963    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:12.744070 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:12.744050    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:12.888776 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:12.888704    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" containerID="bc1c182bd657c8f7a4d2e0e56e1278f318b77fdbdc1f5ae3033207d84355bc09" exitCode=0
Nov 04 07:15:12.889657 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:12.889357    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k7449" event={"ID":"0d74bcca-3fbc-4984-8acf-b53b0f34e30a","Type":"ContainerDied","Data":"bc1c182bd657c8f7a4d2e0e56e1278f318b77fdbdc1f5ae3033207d84355bc09"}
Nov 04 07:15:13.896185 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:13.895540    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-knhlk" event={"ID":"13da5ce6-d81c-49c2-b92b-ffb77d6a2e38","Type":"ContainerStarted","Data":"d7efcb4ed87499b8bdd698da39ba732586b79b7c60da234cf1017fd2ce5426d0"}
Nov 04 07:15:14.741527 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:14.741452    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:14.741527 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:14.741480    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:14.741711 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:14.741550    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:14.741711 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:14.741613    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:14.741822 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:14.741709    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:14.741822 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:14.741800    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:14.905685 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:14.905654    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pxh9" event={"ID":"3f1d3530-4c03-4633-92b1-9fc2ab954b9f","Type":"ContainerStarted","Data":"28fed7ff8711c9a3c10a4ad05bf122c86d6434ef490afa46c5c30770f5a18ba7"}
Nov 04 07:15:14.925921 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:14.925868    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-knhlk" podStartSLOduration=3.79287622 podStartE2EDuration="7.925849433s" totalImagesPullingTime="4.132973213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:13.91491675 +0000 UTC m=+7.725709611" watchObservedRunningTime="2026-11-04 07:15:14.925849433 +0000 UTC m=+8.736642297"
Nov 04 07:15:15.912035 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:15.911994    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m" event={"ID":"07aa084c-c07a-4901-800d-a3ba2a659bd5","Type":"ContainerStarted","Data":"aebda92d80ba3d8d302fcfa3cc35d57c2f8008b7409c6800a3d5eb853dab35d4"}
Nov 04 07:15:16.286958 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.286413    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:15:16.287716 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.287685    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:15:16.311546 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.311415    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2pxh9" podStartSLOduration=3.91359671 podStartE2EDuration="9.31139783s" totalImagesPullingTime="5.39780112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:14.927023543 +0000 UTC m=+8.737816405" watchObservedRunningTime="2026-11-04 07:15:16.31139783 +0000 UTC m=+10.122190690"
Nov 04 07:15:16.365552 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.365529    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48kkb\" (UniqueName: \"kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb\") pod \"network-check-target-smfct\" (UID: \"ab0e4439-974e-420d-be04-e9e86c18d606\") " pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:16.365658 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.365566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:16.365658 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.365614    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:16.365778 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:16.365706    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:16.365778 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:16.365767    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret podName:bd7ea631-0ca9-4829-8408-8352e1d1bc7f nodeName:}" failed. No retries permitted until 2026-11-04 07:15:24.365748149 +0000 UTC m=+18.176540987 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret") pod "global-pull-secret-syncer-dxhns" (UID: "bd7ea631-0ca9-4829-8408-8352e1d1bc7f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:16.365926 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:16.365893    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:16.366006 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:16.365953    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs podName:7a10a31c-6c0e-45e1-9d6f-e1532180e474 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:24.365935776 +0000 UTC m=+18.176728616 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs") pod "network-metrics-daemon-kr5j5" (UID: "7a10a31c-6c0e-45e1-9d6f-e1532180e474") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:16.366074 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:16.366036    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 07:15:16.366074 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:16.366051    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 07:15:16.366074 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:16.366064    2581 projected.go:196] Error preparing data for projected volume kube-api-access-48kkb for pod openshift-network-diagnostics/network-check-target-smfct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:16.366196 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:16.366105    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb podName:ab0e4439-974e-420d-be04-e9e86c18d606 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:24.36609159 +0000 UTC m=+18.176884444 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-48kkb" (UniqueName: "kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb") pod "network-check-target-smfct" (UID: "ab0e4439-974e-420d-be04-e9e86c18d606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:16.505189 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.505153    2581 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 07:15:16.636126 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.635970    2581 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T07:15:16.505182783Z","UUID":"17fd7eec-f532-4636-8f95-e93e44e52930","Handler":null,"Name":"","Endpoint":""}
Nov 04 07:15:16.640574 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.640552    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 07:15:16.640688 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.640591    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 07:15:16.748478 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.748440    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:16.748662 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:16.748584    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:16.749005 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.748981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:16.749115 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:16.749089    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:16.749187 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.749170    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:16.749293 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:16.749269    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:16.919302 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:16.919190    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m" event={"ID":"07aa084c-c07a-4901-800d-a3ba2a659bd5","Type":"ContainerStarted","Data":"67eb4596d097a72bbb3f5404df74d7c1aca30f3ad6c0e6b8151d17406b382ffe"}
Nov 04 07:15:17.923729 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:17.922991    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2zd55" event={"ID":"f95c6e6a-6787-423e-8129-a62feb0a5156","Type":"ContainerStarted","Data":"8dbb743b7b56c6e8f96f7185afddac1bc900c0daaffb7acc22ac15f2b384b394"}
Nov 04 07:15:17.925873 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:17.925840    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" containerID="2f59d8a49e023b3eaf2890a42af41459fdbbadb4ca4aad81da75080bce7436a9" exitCode=0
Nov 04 07:15:17.925981 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:17.925927    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k7449" event={"ID":"0d74bcca-3fbc-4984-8acf-b53b0f34e30a","Type":"ContainerDied","Data":"2f59d8a49e023b3eaf2890a42af41459fdbbadb4ca4aad81da75080bce7436a9"}
Nov 04 07:15:17.929999 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:17.929626    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dn8mz" event={"ID":"caee6966-5544-44a7-8b90-c431ef6b94de","Type":"ContainerStarted","Data":"ef6334439f59fd03942bf9184ec119e981c43a572e8f3cfc3d66e98e25f2e80e"}
Nov 04 07:15:17.932444 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:17.932419    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m" event={"ID":"07aa084c-c07a-4901-800d-a3ba2a659bd5","Type":"ContainerStarted","Data":"8c287b72ae01b11ba03c1b330aa401d6311f60db435d77b7f1aab19d6fbbb349"}
Nov 04 07:15:17.967951 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:17.967900    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-2zd55" podStartSLOduration=2.524861784 podStartE2EDuration="10.967881107s" totalImagesPullingTime="8.443019323s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:17.941512106 +0000 UTC m=+11.752304969" watchObservedRunningTime="2026-11-04 07:15:17.967881107 +0000 UTC m=+11.778673970"
Nov 04 07:15:17.988640 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:17.988413    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m" podStartSLOduration=2.628932506 podStartE2EDuration="10.98839595s" totalImagesPullingTime="8.359463444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:17.98770028 +0000 UTC m=+11.798493140" watchObservedRunningTime="2026-11-04 07:15:17.98839595 +0000 UTC m=+11.799188811"
Nov 04 07:15:18.009133 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:18.009088    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-dn8mz" podStartSLOduration=2.975292945 podStartE2EDuration="11.009072132s" totalImagesPullingTime="8.033779187s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:18.008445502 +0000 UTC m=+11.819238362" watchObservedRunningTime="2026-11-04 07:15:18.009072132 +0000 UTC m=+11.819864987"
Nov 04 07:15:18.740401 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:18.740022    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:18.740401 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:18.740054    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:18.740401 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:18.740329    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:18.740655 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:18.740106    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:18.740655 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:18.740432    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:18.740655 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:18.740520    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:18.936266 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:18.936188    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qt2pw" event={"ID":"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6","Type":"ContainerStarted","Data":"fa2a57a88dca7dcf787de5c31d828b2bda25ac7c488b08ecdb9ab475313cbdc9"}
Nov 04 07:15:18.954951 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:18.954894    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qt2pw" podStartSLOduration=3.850536881 podStartE2EDuration="11.954875107s" totalImagesPullingTime="8.104338226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:18.954085907 +0000 UTC m=+12.764878766" watchObservedRunningTime="2026-11-04 07:15:18.954875107 +0000 UTC m=+12.765667968"
Nov 04 07:15:19.940749 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:19.940664    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-758f6" event={"ID":"03300026-6052-474a-a6b0-53afd4f3f67a","Type":"ContainerStarted","Data":"73fba32b7e2ae29dc7202864b6a0b1df9455a0fa2b26fbab15891427d601bbf4"}
Nov 04 07:15:19.943219 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:19.943189    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" containerID="f928208fba055cd107539cfcf588fa4c5d7d51ba3a89a236864a1a10ede1d97a" exitCode=0
Nov 04 07:15:19.943311 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:19.943221    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k7449" event={"ID":"0d74bcca-3fbc-4984-8acf-b53b0f34e30a","Type":"ContainerDied","Data":"f928208fba055cd107539cfcf588fa4c5d7d51ba3a89a236864a1a10ede1d97a"}
Nov 04 07:15:19.961319 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:19.961273    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-758f6" podStartSLOduration=2.296382803 podStartE2EDuration="12.961262407s" totalImagesPullingTime="10.664879604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:19.961104695 +0000 UTC m=+13.771897555" watchObservedRunningTime="2026-11-04 07:15:19.961262407 +0000 UTC m=+13.772055265"
Nov 04 07:15:20.739580 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:20.739549    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:20.739765 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:20.739613    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:20.739765 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:20.739629    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:20.739765 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:20.739712    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:20.739934 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:20.739822    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:20.739934 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:20.739910    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:21.949635 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:21.949599    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" containerID="e8628a47d3435227018be4523640ff82395540e5c1c7b379ec3cdb62512c3c97" exitCode=0
Nov 04 07:15:21.950282 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:21.949668    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k7449" event={"ID":"0d74bcca-3fbc-4984-8acf-b53b0f34e30a","Type":"ContainerDied","Data":"e8628a47d3435227018be4523640ff82395540e5c1c7b379ec3cdb62512c3c97"}
Nov 04 07:15:22.740090 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:22.740050    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:22.740265 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:22.740101    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:22.740265 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:22.740179    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:22.740265 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:22.740231    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:22.740426 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:22.740330    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:22.740459 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:22.740444    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:23.955406 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:23.955224    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm" event={"ID":"542b9c39-a718-4064-9640-5123b2dcb358","Type":"ContainerStarted","Data":"38ab8deaf85f0c0b4f5cb1f9da8e4c85b543fe05b6b0b72edb2e72f5d0c85c86"}
Nov 04 07:15:23.955760 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:23.955458    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm" event={"ID":"542b9c39-a718-4064-9640-5123b2dcb358","Type":"ContainerStarted","Data":"0f640fb3108624bd7ef0c03fd4c03e418e875cfc696447b02fbdbbb767021e06"}
Nov 04 07:15:23.955760 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:23.955497    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm" event={"ID":"542b9c39-a718-4064-9640-5123b2dcb358","Type":"ContainerStarted","Data":"1bf362650568a9347d2b4988b1a6707e680e1327739d55b4b4b3f18e376ff982"}
Nov 04 07:15:23.955760 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:23.955534    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm" event={"ID":"542b9c39-a718-4064-9640-5123b2dcb358","Type":"ContainerStarted","Data":"f2c1f5343675852cdd1704c02fb29052e43f1966f989f5d024af90ca0a5c8377"}
Nov 04 07:15:23.955760 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:23.955573    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm" event={"ID":"542b9c39-a718-4064-9640-5123b2dcb358","Type":"ContainerStarted","Data":"80e5e0f7e43e2cb2841012143e41cc2f2fde1ec415d99fb5909ab0aa321b791d"}
Nov 04 07:15:23.957862 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:23.957840    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" containerID="0037e001f0bd5a3e3bad7a3ca02ad05415743f76161422ba1ed0b15c4564af8d" exitCode=0
Nov 04 07:15:23.957938 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:23.957878    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k7449" event={"ID":"0d74bcca-3fbc-4984-8acf-b53b0f34e30a","Type":"ContainerDied","Data":"0037e001f0bd5a3e3bad7a3ca02ad05415743f76161422ba1ed0b15c4564af8d"}
Nov 04 07:15:24.423202 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:24.422994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:24.423202 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:24.423171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48kkb\" (UniqueName: \"kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb\") pod \"network-check-target-smfct\" (UID: \"ab0e4439-974e-420d-be04-e9e86c18d606\") " pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:24.423202 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:24.423188    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:24.423421 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:24.423114    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:24.423421 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:24.423264    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret podName:bd7ea631-0ca9-4829-8408-8352e1d1bc7f nodeName:}" failed. No retries permitted until 2026-11-04 07:15:40.423248987 +0000 UTC m=+34.234041824 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret") pod "global-pull-secret-syncer-dxhns" (UID: "bd7ea631-0ca9-4829-8408-8352e1d1bc7f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:24.423421 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:24.423273    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:24.423421 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:24.423326    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs podName:7a10a31c-6c0e-45e1-9d6f-e1532180e474 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:40.423308753 +0000 UTC m=+34.234101606 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs") pod "network-metrics-daemon-kr5j5" (UID: "7a10a31c-6c0e-45e1-9d6f-e1532180e474") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:24.423421 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:24.423330    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 07:15:24.423421 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:24.423348    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 07:15:24.423421 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:24.423357    2581 projected.go:196] Error preparing data for projected volume kube-api-access-48kkb for pod openshift-network-diagnostics/network-check-target-smfct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:24.423421 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:24.423414    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb podName:ab0e4439-974e-420d-be04-e9e86c18d606 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:40.423404553 +0000 UTC m=+34.234197393 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-48kkb" (UniqueName: "kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb") pod "network-check-target-smfct" (UID: "ab0e4439-974e-420d-be04-e9e86c18d606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:24.740053 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:24.739970    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:24.740053 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:24.739999    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:24.740254 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:24.740068    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:24.740254 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:24.740148    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:24.740254 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:24.740243    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:24.740436 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:24.740351    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:24.962658 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:24.962630    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm" event={"ID":"542b9c39-a718-4064-9640-5123b2dcb358","Type":"ContainerStarted","Data":"a585f4516cf1012f07645132ab857c2ace2c14b7be0470e118fa0c370f1d0d6d"}
Nov 04 07:15:24.965410 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:24.965383    2581 generic.go:356] "Generic (PLEG): container finished" podID="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" containerID="62ddb9344a41127f3e6c363a0260ce36b59bbbd5baa3916839099d44f8725204" exitCode=0
Nov 04 07:15:24.965521 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:24.965425    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k7449" event={"ID":"0d74bcca-3fbc-4984-8acf-b53b0f34e30a","Type":"ContainerDied","Data":"62ddb9344a41127f3e6c363a0260ce36b59bbbd5baa3916839099d44f8725204"}
Nov 04 07:15:25.554560 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:25.554531    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2zd55_f95c6e6a-6787-423e-8129-a62feb0a5156/dns-node-resolver/0.log"
Nov 04 07:15:25.970561 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:25.970479    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k7449" event={"ID":"0d74bcca-3fbc-4984-8acf-b53b0f34e30a","Type":"ContainerStarted","Data":"8c6c3ade8635e699d35bd16d26e1458a4a6a23338a0b24e1f91152e52ab31769"}
Nov 04 07:15:25.998063 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:25.998027    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k7449" podStartSLOduration=7.676613853 podStartE2EDuration="18.998014756s" totalImagesPullingTime="11.321400903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:25.997592891 +0000 UTC m=+19.808385749" watchObservedRunningTime="2026-11-04 07:15:25.998014756 +0000 UTC m=+19.808807614"
Nov 04 07:15:26.523241 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:26.523214    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2pxh9_3f1d3530-4c03-4633-92b1-9fc2ab954b9f/node-ca/0.log"
Nov 04 07:15:26.740196 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:26.740170    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:26.740357 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:26.740250    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:26.740420 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:26.740354    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:26.740500 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:26.740478    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:26.740593 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:26.740516    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:26.740630 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:26.740603    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:26.976177 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:26.976146    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm" event={"ID":"542b9c39-a718-4064-9640-5123b2dcb358","Type":"ContainerStarted","Data":"2a19b19eb222b45fe991dddeeffb77de19904eb442c46490f36143c330c6935f"}
Nov 04 07:15:28.740045 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:28.739871    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:28.740608 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:28.739901    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:28.740608 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:28.740119    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:28.740608 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:28.739907    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:28.740608 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:28.740210    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:28.740608 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:28.740296    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:28.984114 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:28.984072    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm" event={"ID":"542b9c39-a718-4064-9640-5123b2dcb358","Type":"ContainerStarted","Data":"c830486e9884e8def2296e97aad80f2f9fe97d0714c96a8c1ab3f333b384e115"}
Nov 04 07:15:28.984378 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:28.984343    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:29.000299 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:29.000219    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:29.019020 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:29.018977    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm" podStartSLOduration=7.452362167 podStartE2EDuration="22.018965097s" totalImagesPullingTime="14.56660293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:29.018673226 +0000 UTC m=+22.829466084" watchObservedRunningTime="2026-11-04 07:15:29.018965097 +0000 UTC m=+22.829757956"
Nov 04 07:15:29.478683 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:29.478649    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:29.986400 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:29.986357    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:30.000705 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:30.000683    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:15:30.739696 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:30.739671    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:30.739835 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:30.739697    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:30.739835 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:30.739730    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:30.739919 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:30.739862    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:30.739977 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:30.739921    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:30.739977 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:30.739952    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:32.739961 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:32.739932    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:32.740404 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:32.739934    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:32.740404 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:32.740052    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:32.740404 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:32.739938    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:32.740404 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:32.740100    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:32.740404 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:32.740158    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:34.739976 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:34.739941    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:34.740470 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:34.740007    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:34.740470 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:34.740005    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:34.740470 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:34.740112    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:34.740470 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:34.740209    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:34.740470 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:34.740274    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:36.312649 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:36.312620    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:15:36.313100 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:36.312733    2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 07:15:36.313177 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:36.313156    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:15:36.740543 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:36.740470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:36.740693 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:36.740549    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:36.740693 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:36.740639    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:36.741203 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:36.741160    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:36.741203 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:36.741176    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:36.743480 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:36.741498    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:38.739718 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:38.739683    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:38.739718 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:38.739721    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:38.740168 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:38.739775    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:38.740168 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:38.739817    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:38.740168 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:38.739904    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:38.740168 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:38.739967    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:40.448548 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:40.448514    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48kkb\" (UniqueName: \"kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb\") pod \"network-check-target-smfct\" (UID: \"ab0e4439-974e-420d-be04-e9e86c18d606\") " pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:40.448548 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:40.448552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:40.449021 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:40.448586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:40.449021 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:40.448663    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 07:15:40.449021 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:40.448684    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 07:15:40.449021 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:40.448694    2581 projected.go:196] Error preparing data for projected volume kube-api-access-48kkb for pod openshift-network-diagnostics/network-check-target-smfct: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:40.449021 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:40.448702    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:40.449021 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:40.448666    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:40.449021 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:40.448736    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb podName:ab0e4439-974e-420d-be04-e9e86c18d606 nodeName:}" failed. No retries permitted until 2026-11-04 07:16:12.448722946 +0000 UTC m=+66.259515797 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-48kkb" (UniqueName: "kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb") pod "network-check-target-smfct" (UID: "ab0e4439-974e-420d-be04-e9e86c18d606") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 07:15:40.449021 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:40.448751    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs podName:7a10a31c-6c0e-45e1-9d6f-e1532180e474 nodeName:}" failed. No retries permitted until 2026-11-04 07:16:12.448745199 +0000 UTC m=+66.259538035 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs") pod "network-metrics-daemon-kr5j5" (UID: "7a10a31c-6c0e-45e1-9d6f-e1532180e474") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 07:15:40.449021 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:40.448762    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret podName:bd7ea631-0ca9-4829-8408-8352e1d1bc7f nodeName:}" failed. No retries permitted until 2026-11-04 07:16:12.448756059 +0000 UTC m=+66.259548897 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret") pod "global-pull-secret-syncer-dxhns" (UID: "bd7ea631-0ca9-4829-8408-8352e1d1bc7f") : object "kube-system"/"original-pull-secret" not registered
Nov 04 07:15:40.739651 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:40.739576    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:40.739651 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:40.739610    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:40.739874 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:40.739576    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:40.739874 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:40.739687    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:40.739874 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:40.739769    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:40.739874 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:40.739853    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:42.740475 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:42.740445    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:42.740914 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:42.740451    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:42.740914 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:42.740568    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:42.740914 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:42.740569    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:42.740914 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:42.740658    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:42.740914 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:42.740708    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:44.742809 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:44.742783    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:44.743194 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:44.742875    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:44.743194 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:44.742882    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:44.743194 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:44.742912    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:44.743194 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:44.742996    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:44.743194 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:44.743041    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:44.898468 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:44.898399    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dxhns"]
Nov 04 07:15:44.902713 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:44.902682    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-smfct"]
Nov 04 07:15:44.902828 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:44.902748    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kr5j5"]
Nov 04 07:15:45.015025 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:45.015001    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:45.015025 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:45.015025    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:45.015192 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:45.015091    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:45.015192 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:45.015124    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:45.015256 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:45.015204    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:45.015346 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:45.015328    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:46.742627 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:46.742461    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:46.743073 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:46.742504    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:46.743073 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:46.742706    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:46.743073 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:46.742513    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:46.743073 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:46.742766    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:46.743073 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:46.742822    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:48.739571 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:48.739541    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:48.739571 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:48.739562    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:48.740116 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:48.739565    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:48.740116 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:48.739639    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-smfct" podUID="ab0e4439-974e-420d-be04-e9e86c18d606"
Nov 04 07:15:48.740116 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:48.739755    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr5j5" podUID="7a10a31c-6c0e-45e1-9d6f-e1532180e474"
Nov 04 07:15:48.740116 ip-10-0-142-197 kubenswrapper[2581]: E1104 07:15:48.739798    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dxhns" podUID="bd7ea631-0ca9-4829-8408-8352e1d1bc7f"
Nov 04 07:15:49.467626 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.467588    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeReady"
Nov 04 07:15:49.467776 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.467714    2581 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 07:15:49.536842 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.536815    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2w47v"]
Nov 04 07:15:49.566012 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.565989    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-f5wrt"]
Nov 04 07:15:49.566116 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.566095    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.576153 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.576107    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6pjn4\""
Nov 04 07:15:49.576279 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.576235    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 07:15:49.577553 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.577540    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 07:15:49.581687 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.581674    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2w47v"]
Nov 04 07:15:49.581733 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.581702    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f5wrt"]
Nov 04 07:15:49.581733 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.581718    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5zbxq"]
Nov 04 07:15:49.581830 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.581797    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f5wrt"
Nov 04 07:15:49.595874 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.595796    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5zbxq"]
Nov 04 07:15:49.595970 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.595928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.611780 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.611757    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/723c3b52-1b88-45aa-9ab1-c0a712bf733e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.611901 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.611796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52e5228a-5720-4bda-9af5-2c1a2d014923-config-volume\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.611901 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.611839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtwxm\" (UniqueName: \"kubernetes.io/projected/52e5228a-5720-4bda-9af5-2c1a2d014923-kube-api-access-rtwxm\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.611967 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.611904    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24419200-a11c-445d-8a5b-d35e6473614f-cert\") pod \"ingress-canary-f5wrt\" (UID: \"24419200-a11c-445d-8a5b-d35e6473614f\") " pod="openshift-ingress-canary/ingress-canary-f5wrt"
Nov 04 07:15:49.611967 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.611936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bwzv\" (UniqueName: \"kubernetes.io/projected/24419200-a11c-445d-8a5b-d35e6473614f-kube-api-access-2bwzv\") pod \"ingress-canary-f5wrt\" (UID: \"24419200-a11c-445d-8a5b-d35e6473614f\") " pod="openshift-ingress-canary/ingress-canary-f5wrt"
Nov 04 07:15:49.612026 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.611967    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqrv7\" (UniqueName: \"kubernetes.io/projected/723c3b52-1b88-45aa-9ab1-c0a712bf733e-kube-api-access-cqrv7\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.612097 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.612042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/723c3b52-1b88-45aa-9ab1-c0a712bf733e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.612127 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.612109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/723c3b52-1b88-45aa-9ab1-c0a712bf733e-data-volume\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.612166 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.612140    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52e5228a-5720-4bda-9af5-2c1a2d014923-tmp-dir\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.612202 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.612181    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/723c3b52-1b88-45aa-9ab1-c0a712bf733e-crio-socket\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.612233 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.612200    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52e5228a-5720-4bda-9af5-2c1a2d014923-metrics-tls\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.677573 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.677556    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 07:15:49.677754 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.677739    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 07:15:49.677819 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.677809    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 07:15:49.677859 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.677839    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 07:15:49.677954 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.677937    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 07:15:49.677989 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.677961    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 07:15:49.677989 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.677980    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-7rfcz\""
Nov 04 07:15:49.679093 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.679075    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 07:15:49.679093 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.679075    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9lks7\""
Nov 04 07:15:49.713395 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.713358    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/723c3b52-1b88-45aa-9ab1-c0a712bf733e-crio-socket\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.713473 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.713411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52e5228a-5720-4bda-9af5-2c1a2d014923-metrics-tls\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.713473 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.713452    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/723c3b52-1b88-45aa-9ab1-c0a712bf733e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.713547 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.713470    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52e5228a-5720-4bda-9af5-2c1a2d014923-config-volume\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.713547 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.713492    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtwxm\" (UniqueName: \"kubernetes.io/projected/52e5228a-5720-4bda-9af5-2c1a2d014923-kube-api-access-rtwxm\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.713547 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.713511    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24419200-a11c-445d-8a5b-d35e6473614f-cert\") pod \"ingress-canary-f5wrt\" (UID: \"24419200-a11c-445d-8a5b-d35e6473614f\") " pod="openshift-ingress-canary/ingress-canary-f5wrt"
Nov 04 07:15:49.713655 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.713572    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/723c3b52-1b88-45aa-9ab1-c0a712bf733e-crio-socket\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.713655 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.713616    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bwzv\" (UniqueName: \"kubernetes.io/projected/24419200-a11c-445d-8a5b-d35e6473614f-kube-api-access-2bwzv\") pod \"ingress-canary-f5wrt\" (UID: \"24419200-a11c-445d-8a5b-d35e6473614f\") " pod="openshift-ingress-canary/ingress-canary-f5wrt"
Nov 04 07:15:49.713655 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.713641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqrv7\" (UniqueName: \"kubernetes.io/projected/723c3b52-1b88-45aa-9ab1-c0a712bf733e-kube-api-access-cqrv7\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.713791 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.713694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/723c3b52-1b88-45aa-9ab1-c0a712bf733e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.713791 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.713721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/723c3b52-1b88-45aa-9ab1-c0a712bf733e-data-volume\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.713791 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.713745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52e5228a-5720-4bda-9af5-2c1a2d014923-tmp-dir\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.726624 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.726599    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52e5228a-5720-4bda-9af5-2c1a2d014923-tmp-dir\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.726705 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.726668    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52e5228a-5720-4bda-9af5-2c1a2d014923-config-volume\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.726705 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.726688    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/723c3b52-1b88-45aa-9ab1-c0a712bf733e-data-volume\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.726705 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.726675    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/723c3b52-1b88-45aa-9ab1-c0a712bf733e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.726950 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.726922    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/723c3b52-1b88-45aa-9ab1-c0a712bf733e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.727146 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.727127    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24419200-a11c-445d-8a5b-d35e6473614f-cert\") pod \"ingress-canary-f5wrt\" (UID: \"24419200-a11c-445d-8a5b-d35e6473614f\") " pod="openshift-ingress-canary/ingress-canary-f5wrt"
Nov 04 07:15:49.728596 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.728572    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52e5228a-5720-4bda-9af5-2c1a2d014923-metrics-tls\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.728872 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.728847    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bwzv\" (UniqueName: \"kubernetes.io/projected/24419200-a11c-445d-8a5b-d35e6473614f-kube-api-access-2bwzv\") pod \"ingress-canary-f5wrt\" (UID: \"24419200-a11c-445d-8a5b-d35e6473614f\") " pod="openshift-ingress-canary/ingress-canary-f5wrt"
Nov 04 07:15:49.728943 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.728930    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqrv7\" (UniqueName: \"kubernetes.io/projected/723c3b52-1b88-45aa-9ab1-c0a712bf733e-kube-api-access-cqrv7\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:49.729073 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.729056    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtwxm\" (UniqueName: \"kubernetes.io/projected/52e5228a-5720-4bda-9af5-2c1a2d014923-kube-api-access-rtwxm\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.902947 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.902876    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:49.920189 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.920165    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f5wrt"
Nov 04 07:15:49.954859 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:49.954828    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:15:50.255264 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.255202    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f5wrt"]
Nov 04 07:15:50.256848 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.256822    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2w47v"]
Nov 04 07:15:50.260124 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:50.260102    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24419200_a11c_445d_8a5b_d35e6473614f.slice/crio-44ee99dbda153808c7164262887164eaaa6fbc4589a4724f11076d13dd0206ad WatchSource:0}: Error finding container 44ee99dbda153808c7164262887164eaaa6fbc4589a4724f11076d13dd0206ad: Status 404 returned error can't find the container with id 44ee99dbda153808c7164262887164eaaa6fbc4589a4724f11076d13dd0206ad
Nov 04 07:15:50.260428 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:50.260407    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e5228a_5720_4bda_9af5_2c1a2d014923.slice/crio-e4179b3bef76fa9449295a47daf08f6d763af821e9e8062124f92049abfce80f WatchSource:0}: Error finding container e4179b3bef76fa9449295a47daf08f6d763af821e9e8062124f92049abfce80f: Status 404 returned error can't find the container with id e4179b3bef76fa9449295a47daf08f6d763af821e9e8062124f92049abfce80f
Nov 04 07:15:50.261590 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.261568    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5zbxq"]
Nov 04 07:15:50.268482 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:50.268457    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723c3b52_1b88_45aa_9ab1_c0a712bf733e.slice/crio-40ade68d52d8cb0c72b6bd3b6e4d90ef5dfda552f677870bfa90d8716044f9c1 WatchSource:0}: Error finding container 40ade68d52d8cb0c72b6bd3b6e4d90ef5dfda552f677870bfa90d8716044f9c1: Status 404 returned error can't find the container with id 40ade68d52d8cb0c72b6bd3b6e4d90ef5dfda552f677870bfa90d8716044f9c1
Nov 04 07:15:50.739671 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.739641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:15:50.739671 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.739657    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:15:50.739943 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.739904    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:15:50.746790 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.746454    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 07:15:50.746790 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.746640    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 07:15:50.746790 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.746716    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-hvg8n\""
Nov 04 07:15:50.747083 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.746938    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zw52p\""
Nov 04 07:15:50.747729 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.747710    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 07:15:50.747860 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.747777    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnthh\""
Nov 04 07:15:50.748090 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:50.748072    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 07:15:51.029689 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.029654    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5zbxq" event={"ID":"723c3b52-1b88-45aa-9ab1-c0a712bf733e","Type":"ContainerStarted","Data":"1d20f0dd023e1ec3ae6eef41e05bb0cf54860f6367a06ae1745f06ef2786f2e8"}
Nov 04 07:15:51.030552 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.029726    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5zbxq" event={"ID":"723c3b52-1b88-45aa-9ab1-c0a712bf733e","Type":"ContainerStarted","Data":"40ade68d52d8cb0c72b6bd3b6e4d90ef5dfda552f677870bfa90d8716044f9c1"}
Nov 04 07:15:51.031160 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.031133    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2w47v" event={"ID":"52e5228a-5720-4bda-9af5-2c1a2d014923","Type":"ContainerStarted","Data":"e4179b3bef76fa9449295a47daf08f6d763af821e9e8062124f92049abfce80f"}
Nov 04 07:15:51.032866 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.032840    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f5wrt" event={"ID":"24419200-a11c-445d-8a5b-d35e6473614f","Type":"ContainerStarted","Data":"44ee99dbda153808c7164262887164eaaa6fbc4589a4724f11076d13dd0206ad"}
Nov 04 07:15:51.538551 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.538513    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"]
Nov 04 07:15:51.550321 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.550288    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.558106 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.558081    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"]
Nov 04 07:15:51.566884 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.566474    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"]
Nov 04 07:15:51.568133 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.567877    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 07:15:51.568133 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.567882    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 04 07:15:51.568133 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.567951    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 07:15:51.568133 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.568030    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 04 07:15:51.568133 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.568107    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 07:15:51.568908 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.568888    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-k5j8w\""
Nov 04 07:15:51.578086 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.578064    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.596084 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.596062    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"]
Nov 04 07:15:51.605745 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.605722    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bwnvs"]
Nov 04 07:15:51.615906 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.615381    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.628554 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-sys\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.628659 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628570    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.628659 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628635    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-metrics-client-ca\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.628743 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628705    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13b111e6-49a0-45e4-bb33-a49c5dc4d439-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.628743 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628736    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.628845 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628771    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/13b111e6-49a0-45e4-bb33-a49c5dc4d439-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.628845 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-accelerators-collector-config\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.628845 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628825    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f79f6\" (UniqueName: \"kubernetes.io/projected/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-kube-api-access-f79f6\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.629062 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-wtmp\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.629062 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628881    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7hnk\" (UniqueName: \"kubernetes.io/projected/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-kube-api-access-q7hnk\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.629062 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-tls\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.629062 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-root\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.629062 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.628974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.629062 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.629009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.629062 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.629046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4fxt\" (UniqueName: \"kubernetes.io/projected/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-api-access-n4fxt\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.629399 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.629087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.629399 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.629117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-textfile\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.629399 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.629158    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.629399 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.629216    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.667804 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.667710    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 07:15:51.667804 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.667726    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 07:15:51.667804 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.667737    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 07:15:51.667804 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.667712    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 04 07:15:51.667804 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.667781    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l2xtx\""
Nov 04 07:15:51.668161 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.667847    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-t6xfx\""
Nov 04 07:15:51.668161 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.667904    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 04 07:15:51.668551 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.668522    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-wtmp\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7hnk\" (UniqueName: \"kubernetes.io/projected/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-kube-api-access-q7hnk\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-tls\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-root\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730404    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730460    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4fxt\" (UniqueName: \"kubernetes.io/projected/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-api-access-n4fxt\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730525    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-textfile\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-sys\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730704    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-metrics-client-ca\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730736    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13b111e6-49a0-45e4-bb33-a49c5dc4d439-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.732248 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.733210 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/13b111e6-49a0-45e4-bb33-a49c5dc4d439-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.733210 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730827    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-accelerators-collector-config\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.733210 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.730856    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f79f6\" (UniqueName: \"kubernetes.io/projected/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-kube-api-access-f79f6\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.733210 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.731315    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-wtmp\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.733210 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.732491    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-root\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.733210 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.732985    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13b111e6-49a0-45e4-bb33-a49c5dc4d439-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.733210 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.733010    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-metrics-client-ca\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.733558 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.733245    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-textfile\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.733558 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.733483    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.734389 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.733961    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/13b111e6-49a0-45e4-bb33-a49c5dc4d439-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.734389 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.734026    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-sys\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.734389 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.734194    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-accelerators-collector-config\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.734935 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.734890    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.740383 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.736398    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.740383 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.736527    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.740383 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.736745    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.740383 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.737677    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-tls\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.740383 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.737814    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.740383 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.737848    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.745012 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.744665    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f79f6\" (UniqueName: \"kubernetes.io/projected/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-kube-api-access-f79f6\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:51.745012 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.744971    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4fxt\" (UniqueName: \"kubernetes.io/projected/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-api-access-n4fxt\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.745352 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.745315    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7hnk\" (UniqueName: \"kubernetes.io/projected/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-kube-api-access-q7hnk\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.890807 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.890771    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:15:51.920571 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.920439    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:15:51.977777 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:51.977742    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:15:52.022270 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:52.022238    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff3c48c_f8c5_4946_a95e_b5faedb4dab8.slice/crio-7d9052f080f1cadd5cbc1adca8b3b37ce84b4b7a6177a3026970461146e91070 WatchSource:0}: Error finding container 7d9052f080f1cadd5cbc1adca8b3b37ce84b4b7a6177a3026970461146e91070: Status 404 returned error can't find the container with id 7d9052f080f1cadd5cbc1adca8b3b37ce84b4b7a6177a3026970461146e91070
Nov 04 07:15:52.037623 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:52.037512    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5zbxq" event={"ID":"723c3b52-1b88-45aa-9ab1-c0a712bf733e","Type":"ContainerStarted","Data":"5347a843f8c1c64ba50c4cf4c1a096cf2224e8b9f01474c0cdf3a5bf31ddd84e"}
Nov 04 07:15:52.039513 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:52.039442    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bwnvs" event={"ID":"fff3c48c-f8c5-4946-a95e-b5faedb4dab8","Type":"ContainerStarted","Data":"7d9052f080f1cadd5cbc1adca8b3b37ce84b4b7a6177a3026970461146e91070"}
Nov 04 07:15:52.190424 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:52.190338    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"]
Nov 04 07:15:52.215748 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:52.215712    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe2e0b0d_3d26_4d4d_90a6_ef499452293b.slice/crio-ac05179d7224574da2fd7d92069268b9575f4ffe0323bffcac63f1df0213cb02 WatchSource:0}: Error finding container ac05179d7224574da2fd7d92069268b9575f4ffe0323bffcac63f1df0213cb02: Status 404 returned error can't find the container with id ac05179d7224574da2fd7d92069268b9575f4ffe0323bffcac63f1df0213cb02
Nov 04 07:15:52.238675 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:52.238650    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"]
Nov 04 07:15:53.045700 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.045520    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd" event={"ID":"13b111e6-49a0-45e4-bb33-a49c5dc4d439","Type":"ContainerStarted","Data":"1d291fe1b129b22f99b2ac41e75ffa3d1eaa271e6e081c6723b685251a1c4859"}
Nov 04 07:15:53.048112 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.048087    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2w47v" event={"ID":"52e5228a-5720-4bda-9af5-2c1a2d014923","Type":"ContainerStarted","Data":"8579ee666f91cda036819eeef320c1b016f99b3554403e2424907c96825d9576"}
Nov 04 07:15:53.048227 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.048133    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2w47v" event={"ID":"52e5228a-5720-4bda-9af5-2c1a2d014923","Type":"ContainerStarted","Data":"fd3dbeb3c8957d925dde488dc7e933659c4dd5c33c43de54b543f499505faddd"}
Nov 04 07:15:53.048973 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.048954    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2w47v"
Nov 04 07:15:53.051341 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.051318    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc" event={"ID":"fe2e0b0d-3d26-4d4d-90a6-ef499452293b","Type":"ContainerStarted","Data":"511c2cf066b5356b45d8bedba3a5a78f43bc5ea2e579b14ba75e4a9e4c69f4e0"}
Nov 04 07:15:53.051441 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.051392    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc" event={"ID":"fe2e0b0d-3d26-4d4d-90a6-ef499452293b","Type":"ContainerStarted","Data":"50bf52d019e72aa0e2321ced9e9b01e6b336e83d42a3d7dc4239213c9aa84b37"}
Nov 04 07:15:53.051441 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.051427    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc" event={"ID":"fe2e0b0d-3d26-4d4d-90a6-ef499452293b","Type":"ContainerStarted","Data":"ac05179d7224574da2fd7d92069268b9575f4ffe0323bffcac63f1df0213cb02"}
Nov 04 07:15:53.700669 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.700628    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-2w47v" podStartSLOduration=2.573867875 podStartE2EDuration="4.700615385s" totalImagesPullingTime="2.12674751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:53.072667766 +0000 UTC m=+46.883460625" watchObservedRunningTime="2026-11-04 07:15:53.700615385 +0000 UTC m=+47.511408244"
Nov 04 07:15:53.701015 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.701002    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-64cb9dd857-hf26d"]
Nov 04 07:15:53.703664 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.703650    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.715588 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.715556    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 04 07:15:53.715959 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.715929    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 04 07:15:53.716034 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.716023    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2tsas6k3v318c\""
Nov 04 07:15:53.716090 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.716080    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 04 07:15:53.716212 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.716195    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-fcnxr\""
Nov 04 07:15:53.716283 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.716238    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 04 07:15:53.716322 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.716292    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 04 07:15:53.757560 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.756472    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-grpc-tls\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.757560 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.756522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.757560 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.756610    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.757560 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.756637    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.757560 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.756679    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.757560 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.756723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-tls\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.757560 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.756779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-tmp\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.757560 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.756817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h52sl\" (UniqueName: \"kubernetes.io/projected/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-kube-api-access-h52sl\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.757560 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.756953    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-metrics-client-ca\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.758206 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.758185    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64cb9dd857-hf26d"]
Nov 04 07:15:53.857823 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.857793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-tls\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.857936 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.857867    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-tmp\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.857936 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.857907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h52sl\" (UniqueName: \"kubernetes.io/projected/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-kube-api-access-h52sl\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.858039 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.857959    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-metrics-client-ca\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.858039 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.857990    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-grpc-tls\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.858039 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.858018    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.858181 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.858054    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.858181 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.858082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.858181 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.858123    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.859535 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.859507    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-tmp\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.861018 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.860970    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-metrics-client-ca\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.862713 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.862688    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.863790 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.863741    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.864967 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.864941    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-grpc-tls\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.865063 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.864986    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.865330 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.865310    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-tls\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.866413 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.866392    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:53.871961 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:53.871933    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h52sl\" (UniqueName: \"kubernetes.io/projected/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-kube-api-access-h52sl\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:54.059115 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:54.059085    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5zbxq" event={"ID":"723c3b52-1b88-45aa-9ab1-c0a712bf733e","Type":"ContainerStarted","Data":"85864f708a51a5375481e793954a3ad472e96dc6a44b79fbb5c61415529a216c"}
Nov 04 07:15:54.060840 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:54.060814    2581 generic.go:356] "Generic (PLEG): container finished" podID="fff3c48c-f8c5-4946-a95e-b5faedb4dab8" containerID="c9f3ee60a8c8fc79c4ff2340f5e070adec8a7b2578ec4fc277e7841a4f040da5" exitCode=0
Nov 04 07:15:54.060949 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:54.060869    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bwnvs" event={"ID":"fff3c48c-f8c5-4946-a95e-b5faedb4dab8","Type":"ContainerDied","Data":"c9f3ee60a8c8fc79c4ff2340f5e070adec8a7b2578ec4fc277e7841a4f040da5"}
Nov 04 07:15:54.061221 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:54.061204    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:54.064600 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:54.064563    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f5wrt" event={"ID":"24419200-a11c-445d-8a5b-d35e6473614f","Type":"ContainerStarted","Data":"2b05772a2d02ead57fbb51fac54a464e8950bfca9db73cb4cf164635d6271bff"}
Nov 04 07:15:54.088508 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:54.088465    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5zbxq" podStartSLOduration=1.886807492 podStartE2EDuration="5.088428281s" totalImagesPullingTime="3.201620789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:54.088136758 +0000 UTC m=+47.898929608" watchObservedRunningTime="2026-11-04 07:15:54.088428281 +0000 UTC m=+47.899221141"
Nov 04 07:15:54.122252 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:54.121793    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-f5wrt" podStartSLOduration=1.702138881 podStartE2EDuration="5.121772416s" totalImagesPullingTime="3.419633535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:54.120812151 +0000 UTC m=+47.931605011" watchObservedRunningTime="2026-11-04 07:15:54.121772416 +0000 UTC m=+47.932565268"
Nov 04 07:15:54.379405 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:54.379352    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64cb9dd857-hf26d"]
Nov 04 07:15:54.383572 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:54.383548    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77e0a9a6_4d92_4f21_8ecf_7b791b33a730.slice/crio-d6873c187f2c08e1a814be480c98c18baf108443511067bf4f4546299ccb20a3 WatchSource:0}: Error finding container d6873c187f2c08e1a814be480c98c18baf108443511067bf4f4546299ccb20a3: Status 404 returned error can't find the container with id d6873c187f2c08e1a814be480c98c18baf108443511067bf4f4546299ccb20a3
Nov 04 07:15:55.070394 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:55.070339    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd" event={"ID":"13b111e6-49a0-45e4-bb33-a49c5dc4d439","Type":"ContainerStarted","Data":"5cd01311a329df046ec253c4bb1bb7e4b6f825928c1336a86fb0fe1c73d19be0"}
Nov 04 07:15:55.070801 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:55.070421    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd" event={"ID":"13b111e6-49a0-45e4-bb33-a49c5dc4d439","Type":"ContainerStarted","Data":"77b13b35d1930b12cb9470e6ad289156fa011c93fd56b9da4dceac0d9ce97310"}
Nov 04 07:15:55.070801 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:55.070449    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd" event={"ID":"13b111e6-49a0-45e4-bb33-a49c5dc4d439","Type":"ContainerStarted","Data":"c3644ff28c8ac9b62f19d1a42dbea8e957f6bcae999f691c525c83c907038989"}
Nov 04 07:15:55.072725 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:55.072695    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc" event={"ID":"fe2e0b0d-3d26-4d4d-90a6-ef499452293b","Type":"ContainerStarted","Data":"89dcd1061c9947c059af9193e0c99dd428b3e792dd2386adbfd334d9d15264a8"}
Nov 04 07:15:55.078353 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:55.078326    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d" event={"ID":"77e0a9a6-4d92-4f21-8ecf-7b791b33a730","Type":"ContainerStarted","Data":"d6873c187f2c08e1a814be480c98c18baf108443511067bf4f4546299ccb20a3"}
Nov 04 07:15:55.080689 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:55.080664    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bwnvs" event={"ID":"fff3c48c-f8c5-4946-a95e-b5faedb4dab8","Type":"ContainerStarted","Data":"a6bad587471573771a4d6ac34ca1efcd5e765e879f99bda04899578d59383c53"}
Nov 04 07:15:55.080782 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:55.080706    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bwnvs" event={"ID":"fff3c48c-f8c5-4946-a95e-b5faedb4dab8","Type":"ContainerStarted","Data":"e6bde2257dae8f1bd081357741730c633dbf0499afa9bd504babf9aa7bbaaaf9"}
Nov 04 07:15:55.095786 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:55.095734    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd" podStartSLOduration=2.12448633 podStartE2EDuration="4.095719153s" totalImagesPullingTime="1.971232823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:55.093498073 +0000 UTC m=+48.904290943" watchObservedRunningTime="2026-11-04 07:15:55.095719153 +0000 UTC m=+48.906512015"
Nov 04 07:15:55.121979 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:55.121434    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc" podStartSLOduration=2.6305975950000002 podStartE2EDuration="4.121417948s" totalImagesPullingTime="1.490820353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:55.120501167 +0000 UTC m=+48.931294048" watchObservedRunningTime="2026-11-04 07:15:55.121417948 +0000 UTC m=+48.932210806"
Nov 04 07:15:55.143277 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:55.143231    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bwnvs" podStartSLOduration=2.426103088 podStartE2EDuration="4.143218121s" totalImagesPullingTime="1.641133925s" totalInitContainerRuntime="75.981108ms" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:55.142442452 +0000 UTC m=+48.953235337" watchObservedRunningTime="2026-11-04 07:15:55.143218121 +0000 UTC m=+48.954010981"
Nov 04 07:15:55.998920 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:55.998885    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-76495f7c59-rsjg4"]
Nov 04 07:15:56.001199 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.001169    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.007919 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.007882    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 07:15:56.015506 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.015466    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76495f7c59-rsjg4"]
Nov 04 07:15:56.036477 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.036447    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 04 07:15:56.037084 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.037038    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2lqsj\""
Nov 04 07:15:56.037084 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.037039    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-erst5rdl7thnl\""
Nov 04 07:15:56.037211 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.037160    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 04 07:15:56.073182 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.073156    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef74e4a-dee2-4554-be57-0eda5677ebf2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.073548 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.073191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-client-ca-bundle\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.073548 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.073208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4hwz\" (UniqueName: \"kubernetes.io/projected/0ef74e4a-dee2-4554-be57-0eda5677ebf2-kube-api-access-g4hwz\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.073548 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.073317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0ef74e4a-dee2-4554-be57-0eda5677ebf2-metrics-server-audit-profiles\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.073548 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.073400    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-secret-metrics-server-tls\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.073548 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.073441    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-secret-metrics-server-client-certs\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.073548 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.073455    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0ef74e4a-dee2-4554-be57-0eda5677ebf2-audit-log\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.111057 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.111027    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 04 07:15:56.174604 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.174561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-secret-metrics-server-tls\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.174774 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.174659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-secret-metrics-server-client-certs\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.174894 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.174867    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0ef74e4a-dee2-4554-be57-0eda5677ebf2-audit-log\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.175103 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.175058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef74e4a-dee2-4554-be57-0eda5677ebf2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.175210 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.175144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-client-ca-bundle\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.175210 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.175174    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4hwz\" (UniqueName: \"kubernetes.io/projected/0ef74e4a-dee2-4554-be57-0eda5677ebf2-kube-api-access-g4hwz\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.175210 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.175184    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0ef74e4a-dee2-4554-be57-0eda5677ebf2-audit-log\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.175387 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.175252    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0ef74e4a-dee2-4554-be57-0eda5677ebf2-metrics-server-audit-profiles\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.175898 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.175869    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef74e4a-dee2-4554-be57-0eda5677ebf2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.176382 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.176338    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0ef74e4a-dee2-4554-be57-0eda5677ebf2-metrics-server-audit-profiles\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.177200 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.177156    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-secret-metrics-server-client-certs\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.177293 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.177274    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-secret-metrics-server-tls\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.177838 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.177814    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-client-ca-bundle\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.188635 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.188611    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4hwz\" (UniqueName: \"kubernetes.io/projected/0ef74e4a-dee2-4554-be57-0eda5677ebf2-kube-api-access-g4hwz\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.303943 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.303916    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"]
Nov 04 07:15:56.307371 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.307342    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:15:56.311315 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.311292    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 04 07:15:56.311418 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.311329    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jpdvn\""
Nov 04 07:15:56.316547 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.316527    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:15:56.317146 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.317123    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"]
Nov 04 07:15:56.377102 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.376972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-2xqz7\" (UID: \"2904aa1a-7fd1-43c5-922a-2a0a85e2ab05\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:15:56.377102 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.377015    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbnh9\" (UniqueName: \"kubernetes.io/projected/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05-kube-api-access-pbnh9\") pod \"monitoring-plugin-77f4f76594-2xqz7\" (UID: \"2904aa1a-7fd1-43c5-922a-2a0a85e2ab05\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:15:56.477472 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.477448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-2xqz7\" (UID: \"2904aa1a-7fd1-43c5-922a-2a0a85e2ab05\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:15:56.477567 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.477492    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbnh9\" (UniqueName: \"kubernetes.io/projected/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05-kube-api-access-pbnh9\") pod \"monitoring-plugin-77f4f76594-2xqz7\" (UID: \"2904aa1a-7fd1-43c5-922a-2a0a85e2ab05\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:15:56.479606 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.479578    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-2xqz7\" (UID: \"2904aa1a-7fd1-43c5-922a-2a0a85e2ab05\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:15:56.489991 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.489969    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbnh9\" (UniqueName: \"kubernetes.io/projected/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05-kube-api-access-pbnh9\") pod \"monitoring-plugin-77f4f76594-2xqz7\" (UID: \"2904aa1a-7fd1-43c5-922a-2a0a85e2ab05\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:15:56.597333 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.597309    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76495f7c59-rsjg4"]
Nov 04 07:15:56.599429 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:56.599405    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef74e4a_dee2_4554_be57_0eda5677ebf2.slice/crio-d71980dfca8b151274ef89063a68a58fa1bb2e0cf015d0f14db65e7d2f013042 WatchSource:0}: Error finding container d71980dfca8b151274ef89063a68a58fa1bb2e0cf015d0f14db65e7d2f013042: Status 404 returned error can't find the container with id d71980dfca8b151274ef89063a68a58fa1bb2e0cf015d0f14db65e7d2f013042
Nov 04 07:15:56.637441 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.637422    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:15:56.909445 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:56.909349    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"]
Nov 04 07:15:56.912253 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:56.912228    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2904aa1a_7fd1_43c5_922a_2a0a85e2ab05.slice/crio-f6efbf2d5da3a0a56cbf1edb25965f87282945eb79bfd9f2664647e8d87d5b52 WatchSource:0}: Error finding container f6efbf2d5da3a0a56cbf1edb25965f87282945eb79bfd9f2664647e8d87d5b52: Status 404 returned error can't find the container with id f6efbf2d5da3a0a56cbf1edb25965f87282945eb79bfd9f2664647e8d87d5b52
Nov 04 07:15:57.090585 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.090405    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d" event={"ID":"77e0a9a6-4d92-4f21-8ecf-7b791b33a730","Type":"ContainerStarted","Data":"5c33750f816f7ee44d92065aea12cb0417e3633c4a15d256d148dfcea25b2989"}
Nov 04 07:15:57.090585 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.090497    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d" event={"ID":"77e0a9a6-4d92-4f21-8ecf-7b791b33a730","Type":"ContainerStarted","Data":"bc394e87f4cdbd42f12a2b04d6b685d3a30a35888725d914c5c62ae5ab2ec6f4"}
Nov 04 07:15:57.090585 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.090538    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d" event={"ID":"77e0a9a6-4d92-4f21-8ecf-7b791b33a730","Type":"ContainerStarted","Data":"91edc5205409ad00b0fb53c340dcc681fc16e253426dd92d537868c827dd7f88"}
Nov 04 07:15:57.092390 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.092347    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7" event={"ID":"2904aa1a-7fd1-43c5-922a-2a0a85e2ab05","Type":"ContainerStarted","Data":"f6efbf2d5da3a0a56cbf1edb25965f87282945eb79bfd9f2664647e8d87d5b52"}
Nov 04 07:15:57.093682 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.093657    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4" event={"ID":"0ef74e4a-dee2-4554-be57-0eda5677ebf2","Type":"ContainerStarted","Data":"d71980dfca8b151274ef89063a68a58fa1bb2e0cf015d0f14db65e7d2f013042"}
Nov 04 07:15:57.958936 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.958898    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 07:15:57.963100 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.963076    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.974662 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.973728    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 04 07:15:57.974662 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.973877    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 04 07:15:57.974662 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.974186    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 04 07:15:57.974662 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.974224    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 04 07:15:57.974662 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.974317    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 04 07:15:57.974662 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.974317    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 04 07:15:57.974662 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.974540    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 04 07:15:57.975179 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.975160    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 04 07:15:57.975434 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.975301    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vb7ng\""
Nov 04 07:15:57.975434 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.975409    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 04 07:15:57.977671 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.977651    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 04 07:15:57.978745 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.978727    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7a9sgmd41l2bu\""
Nov 04 07:15:57.978745 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.978727    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 04 07:15:57.978895 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.978751    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 04 07:15:57.990290 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.990264    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 04 07:15:57.993218 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993195    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993321 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993321 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993463 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993463 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993380    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993463 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993602 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993467    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993602 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993602 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993706 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993706 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993706 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993665    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxgcd\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-kube-api-access-mxgcd\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993805 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993805 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993739    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993805 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993903 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993811    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config-out\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993903 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-web-config\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.993903 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993897    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.994023 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993919    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.994023 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.993961    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:57.996782 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:57.996751    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 04 07:15:58.000795 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.000765    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 07:15:58.094805 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.094777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-web-config\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095169 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.094841    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095169 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.094876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095169 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.094924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095169 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.094951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095169 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.094972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095169 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095169 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095169 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095056    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095169 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095084    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095169 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095169 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095169 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095167    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095736 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095192    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095736 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095215    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095736 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095238    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxgcd\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-kube-api-access-mxgcd\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095736 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095281    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095736 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095736 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.095736 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.095383    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config-out\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.096352 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.096330    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.097395 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.097355    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.097643 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.097617    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.099485 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.098535    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.099485 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.099164    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-web-config\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.099485 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.099356    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.099485 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.099422    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.099721 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.099601    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config-out\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.100030 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.100000    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.100642 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.100616    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.100733 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.100632    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.101920 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.101893    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.102281 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.102246    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.102679 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.102654    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.102977 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.102958    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.103109 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.103091    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.103317 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.103293    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.103479 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.103399    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.103566 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.103549    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.107755 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.107729    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d" event={"ID":"77e0a9a6-4d92-4f21-8ecf-7b791b33a730","Type":"ContainerStarted","Data":"f6aad4abeea6eca89f13c43796891ae828f45f7a9f14f8d8715d2e1712d83c01"}
Nov 04 07:15:58.107860 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.107852    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d" event={"ID":"77e0a9a6-4d92-4f21-8ecf-7b791b33a730","Type":"ContainerStarted","Data":"e5a2cd789eb5fe94b45dc9e584d3be74dfb70098145a55350cb269ea7b4412ee"}
Nov 04 07:15:58.107897 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.107879    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d" event={"ID":"77e0a9a6-4d92-4f21-8ecf-7b791b33a730","Type":"ContainerStarted","Data":"bf6e5e91e5d9ba48e43cddd7e1fc4265c9966b074feefd3f562fec504ce44b9b"}
Nov 04 07:15:58.107972 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.107956    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:15:58.115033 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.114993    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxgcd\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-kube-api-access-mxgcd\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.135868 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.135799    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d" podStartSLOduration=2.496558149 podStartE2EDuration="5.135782435s" totalImagesPullingTime="2.639224286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:58.135417133 +0000 UTC m=+51.946209995" watchObservedRunningTime="2026-11-04 07:15:58.135782435 +0000 UTC m=+51.946575278"
Nov 04 07:15:58.291816 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.291786    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:15:58.607118 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:58.607070    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 07:15:58.609431 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:15:58.609403    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb89bad1_c644_4d86_b4bf_0827628ac4d3.slice/crio-6d63d36dfd91395a2a336c91682cff45aea0eba596c6c8e305a4a618d84bdb46 WatchSource:0}: Error finding container 6d63d36dfd91395a2a336c91682cff45aea0eba596c6c8e305a4a618d84bdb46: Status 404 returned error can't find the container with id 6d63d36dfd91395a2a336c91682cff45aea0eba596c6c8e305a4a618d84bdb46
Nov 04 07:15:59.113888 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:59.113855    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7" event={"ID":"2904aa1a-7fd1-43c5-922a-2a0a85e2ab05","Type":"ContainerStarted","Data":"2c766ef38a6cd2723e5ebb74994a62654978479b81efd0b8383926a39bef06bd"}
Nov 04 07:15:59.114331 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:59.114020    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:15:59.115738 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:59.115700    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4" event={"ID":"0ef74e4a-dee2-4554-be57-0eda5677ebf2","Type":"ContainerStarted","Data":"0783e8f800182b5aeb66a9c56484b5955be304316ab78800ac103a04321e750a"}
Nov 04 07:15:59.117225 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:59.117202    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerStarted","Data":"6d63d36dfd91395a2a336c91682cff45aea0eba596c6c8e305a4a618d84bdb46"}
Nov 04 07:15:59.119192 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:59.119172    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:15:59.156338 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:59.156279    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7" podStartSLOduration=1.7472632940000001 podStartE2EDuration="3.156263111s" totalImagesPullingTime="1.408999817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:59.13350775 +0000 UTC m=+52.944300607" watchObservedRunningTime="2026-11-04 07:15:59.156263111 +0000 UTC m=+52.967055971"
Nov 04 07:15:59.157535 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:15:59.157491    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4" podStartSLOduration=2.681791101 podStartE2EDuration="4.157431541s" totalImagesPullingTime="1.47564044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:59.155745647 +0000 UTC m=+52.966538512" watchObservedRunningTime="2026-11-04 07:15:59.157431541 +0000 UTC m=+52.968224399"
Nov 04 07:16:00.122129 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:00.122093    2581 generic.go:356] "Generic (PLEG): container finished" podID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerID="37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7" exitCode=0
Nov 04 07:16:00.122559 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:00.122181    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerDied","Data":"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"}
Nov 04 07:16:02.003045 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:02.003010    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:03.144786 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:03.144686    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerStarted","Data":"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"}
Nov 04 07:16:03.145681 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:03.145658    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerStarted","Data":"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"}
Nov 04 07:16:03.145846 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:03.145830    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerStarted","Data":"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"}
Nov 04 07:16:04.082965 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:04.082936    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2w47v"
Nov 04 07:16:04.123791 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:04.123758    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:04.152147 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:04.152117    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerStarted","Data":"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"}
Nov 04 07:16:04.152518 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:04.152181    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerStarted","Data":"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"}
Nov 04 07:16:04.152518 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:04.152215    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerStarted","Data":"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"}
Nov 04 07:16:08.292225 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:08.292194    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:12.515507 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.515470    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:16:12.515978 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.515522    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:16:12.515978 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.515586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48kkb\" (UniqueName: \"kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb\") pod \"network-check-target-smfct\" (UID: \"ab0e4439-974e-420d-be04-e9e86c18d606\") " pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:16:12.518754 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.518735    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 07:16:12.519867 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.519849    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 07:16:12.520351 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.520339    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 07:16:12.528640 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.528620    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:16:12.528721 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.528653    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:16:12.529332 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.529318    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 07:16:12.539560 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.539542    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48kkb\" (UniqueName: \"kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb\") pod \"network-check-target-smfct\" (UID: \"ab0e4439-974e-420d-be04-e9e86c18d606\") " pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:16:12.669324 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.669289    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zw52p\""
Nov 04 07:16:12.676521 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.676498    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:16:12.681809 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.681793    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-hvg8n\""
Nov 04 07:16:12.689081 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.689049    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:16:12.715282 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.715254    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnthh\""
Nov 04 07:16:12.723018 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.722986    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:16:12.949921 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.949874    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=12.206361306 podStartE2EDuration="15.94985795s" totalImagesPullingTime="3.727834911s" totalInitContainerRuntime="15.661733ms" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:16:04.221099954 +0000 UTC m=+58.031892808" watchObservedRunningTime="2026-11-04 07:16:12.94985795 +0000 UTC m=+66.760650851"
Nov 04 07:16:12.950506 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.950484    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dxhns"]
Nov 04 07:16:12.952348 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:16:12.952319    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7ea631_0ca9_4829_8408_8352e1d1bc7f.slice/crio-b191bb4676bff161861b6bc9b7f749df12b41429017cd23c1505ade498400946 WatchSource:0}: Error finding container b191bb4676bff161861b6bc9b7f749df12b41429017cd23c1505ade498400946: Status 404 returned error can't find the container with id b191bb4676bff161861b6bc9b7f749df12b41429017cd23c1505ade498400946
Nov 04 07:16:12.970815 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:12.970748    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-smfct"]
Nov 04 07:16:12.973289 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:16:12.973256    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab0e4439_974e_420d_be04_e9e86c18d606.slice/crio-f5d07436b33e831ddf14cc9ad79f505b3aed34f0b09d534370911f772f0b6817 WatchSource:0}: Error finding container f5d07436b33e831ddf14cc9ad79f505b3aed34f0b09d534370911f772f0b6817: Status 404 returned error can't find the container with id f5d07436b33e831ddf14cc9ad79f505b3aed34f0b09d534370911f772f0b6817
Nov 04 07:16:13.021617 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:13.021593    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kr5j5"]
Nov 04 07:16:13.023964 ip-10-0-142-197 kubenswrapper[2581]: W1104 07:16:13.023939    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a10a31c_6c0e_45e1_9d6f_e1532180e474.slice/crio-c4d85dd29266327e7e78a085190698c87f6059d9338a9b0cfbedc52aeab52545 WatchSource:0}: Error finding container c4d85dd29266327e7e78a085190698c87f6059d9338a9b0cfbedc52aeab52545: Status 404 returned error can't find the container with id c4d85dd29266327e7e78a085190698c87f6059d9338a9b0cfbedc52aeab52545
Nov 04 07:16:13.183333 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:13.183248    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dxhns" event={"ID":"bd7ea631-0ca9-4829-8408-8352e1d1bc7f","Type":"ContainerStarted","Data":"b191bb4676bff161861b6bc9b7f749df12b41429017cd23c1505ade498400946"}
Nov 04 07:16:13.184470 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:13.184439    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kr5j5" event={"ID":"7a10a31c-6c0e-45e1-9d6f-e1532180e474","Type":"ContainerStarted","Data":"c4d85dd29266327e7e78a085190698c87f6059d9338a9b0cfbedc52aeab52545"}
Nov 04 07:16:13.185478 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:13.185456    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-smfct" event={"ID":"ab0e4439-974e-420d-be04-e9e86c18d606","Type":"ContainerStarted","Data":"f5d07436b33e831ddf14cc9ad79f505b3aed34f0b09d534370911f772f0b6817"}
Nov 04 07:16:15.202880 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:15.202834    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kr5j5" event={"ID":"7a10a31c-6c0e-45e1-9d6f-e1532180e474","Type":"ContainerStarted","Data":"592c608354e1a8c16492b0f6f313f57a241d14adae4ed1fac3b65e88378d9c72"}
Nov 04 07:16:15.203283 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:15.202904    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kr5j5" event={"ID":"7a10a31c-6c0e-45e1-9d6f-e1532180e474","Type":"ContainerStarted","Data":"4470dc39885c41097b17fc007125ef2dc1be5fdd22259daad1b15eedaca22c82"}
Nov 04 07:16:15.224935 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:15.224879    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kr5j5" podStartSLOduration=67.068688899 podStartE2EDuration="1m8.224860715s" totalImagesPullingTime="1.156171816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:16:15.223119295 +0000 UTC m=+69.033912154" watchObservedRunningTime="2026-11-04 07:16:15.224860715 +0000 UTC m=+69.035653576"
Nov 04 07:16:16.317616 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:16.317589    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:16.318510 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:16.318486    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:17.217997 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:17.217962    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-smfct" event={"ID":"ab0e4439-974e-420d-be04-e9e86c18d606","Type":"ContainerStarted","Data":"9aaacf886b94d58d37086d52537b3208f4c14f0a2d4f923bbb464d475f9ad4b1"}
Nov 04 07:16:17.218182 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:17.218058    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:16:17.239901 ip-10-0-142-197 kubenswrapper[2581]: I1104 07:16:17.239848    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-smfct" podStartSLOduration=66.938981217 podStartE2EDuration="1m10.239813951s" totalImagesPullingTime="3.300832734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:16:17.238480232 +0000 UTC m=+71.049273090" watchObservedRunningTime="2026-11-04 07:16:17.239813951 +0000 UTC m=+71.050606810"
Nov 04 07:16:17.913151 ip-10-0-142-197 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 07:16:17.928733 ip-10-0-142-197 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 07:16:17.928901 ip-10-0-142-197 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 07:16:17.929083 ip-10-0-142-197 systemd[1]: kubelet.service: Consumed 4.072s CPU time.
Nov 04 07:16:17.939145 ip-10-0-142-197 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 07:16:18.009455 ip-10-0-142-197 kubenswrapper[5626]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 07:16:18.009455 ip-10-0-142-197 kubenswrapper[5626]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 07:16:18.009455 ip-10-0-142-197 kubenswrapper[5626]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 07:16:18.009455 ip-10-0-142-197 kubenswrapper[5626]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 07:16:18.011428 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011331    5626 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:16:18.011428 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011428    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011433    5626 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011439    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011442    5626 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011445    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011448    5626 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011451    5626 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011454    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011456    5626 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011459    5626 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011461    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011464    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011467    5626 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011469    5626 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011471    5626 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011474    5626 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011477    5626 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011479    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011482    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011490    5626 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:16:18.011497 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011493    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011496    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011499    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011501    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011504    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011507    5626 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011511    5626 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011515    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011518    5626 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011520    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011523    5626 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011525    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011528    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011530    5626 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011533    5626 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011536    5626 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011538    5626 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011541    5626 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011544    5626 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:16:18.012034 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011546    5626 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011549    5626 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011551    5626 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011554    5626 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011556    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011559    5626 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011561    5626 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011564    5626 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011567    5626 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011570    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011572    5626 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011575    5626 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011577    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011580    5626 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011583    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011585    5626 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011588    5626 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011592    5626 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011594    5626 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011597    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:16:18.012541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011599    5626 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011602    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011604    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011607    5626 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011610    5626 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011612    5626 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011615    5626 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011617    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011620    5626 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011622    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011625    5626 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011627    5626 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011631    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011634    5626 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011636    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011641    5626 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011644    5626 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011646    5626 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011649    5626 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011652    5626 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:16:18.013060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011655    5626 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011657    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011659    5626 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011662    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011665    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011668    5626 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011671    5626 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011674    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011677    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011680    5626 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011683    5626 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011685    5626 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011688    5626 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011690    5626 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011693    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011695    5626 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011697    5626 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011700    5626 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011703    5626 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:16:18.013656 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011705    5626 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011708    5626 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011710    5626 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011713    5626 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011715    5626 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011718    5626 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.011721    5626 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012651    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012661    5626 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012665    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012668    5626 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012671    5626 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012673    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012676    5626 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012679    5626 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012681    5626 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012684    5626 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012686    5626 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012689    5626 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012692    5626 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012695    5626 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:16:18.014123 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012698    5626 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012701    5626 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012703    5626 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012706    5626 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012708    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012711    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012714    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012717    5626 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012721    5626 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012725    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012728    5626 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012733    5626 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012738    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012741    5626 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012743    5626 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012747    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012749    5626 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012752    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012755    5626 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:16:18.014687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012758    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012761    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012764    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012766    5626 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012769    5626 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012772    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012774    5626 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012777    5626 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012779    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012782    5626 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012784    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012787    5626 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012789    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012792    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012794    5626 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012797    5626 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012800    5626 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012802    5626 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012805    5626 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012809    5626 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:16:18.015148 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012813    5626 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012817    5626 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012820    5626 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012824    5626 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012827    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012830    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012833    5626 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012836    5626 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012839    5626 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012842    5626 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012845    5626 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012848    5626 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012851    5626 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012854    5626 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012856    5626 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012859    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012861    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012864    5626 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012866    5626 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:16:18.015633 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012869    5626 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012871    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012876    5626 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012880    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012885    5626 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012888    5626 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012892    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012894    5626 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012897    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012900    5626 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012903    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012906    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012908    5626 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012911    5626 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012914    5626 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012916    5626 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012919    5626 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012921    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012924    5626 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:16:18.016161 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012926    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012929    5626 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012932    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012934    5626 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012937    5626 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012942    5626 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012944    5626 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012948    5626 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012950    5626 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012953    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012957    5626 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012959    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012962    5626 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012964    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.012967    5626 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013454    5626 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013461    5626 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013465    5626 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013470    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:16:18.016673 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013473    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013476    5626 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013478    5626 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013482    5626 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013484    5626 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013487    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013490    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013492    5626 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013495    5626 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013498    5626 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013500    5626 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013503    5626 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013505    5626 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013508    5626 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013511    5626 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013513    5626 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013515    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013518    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013522    5626 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013525    5626 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:16:18.017175 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013528    5626 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013531    5626 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013533    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013536    5626 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013538    5626 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013541    5626 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013544    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013546    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013548    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013551    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013553    5626 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013556    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013558    5626 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013561    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013564    5626 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013566    5626 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013569    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013571    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013574    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:16:18.017681 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013576    5626 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013579    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013582    5626 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013584    5626 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013586    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013589    5626 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013592    5626 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013595    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013597    5626 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013600    5626 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013602    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013607    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013609    5626 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013612    5626 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013615    5626 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013617    5626 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013621    5626 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013623    5626 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013625    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013628    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:16:18.018144 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013630    5626 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013633    5626 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013636    5626 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013638    5626 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013642    5626 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013645    5626 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013647    5626 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013650    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013652    5626 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013655    5626 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013658    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013660    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013663    5626 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013666    5626 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013668    5626 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013671    5626 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013673    5626 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013676    5626 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013678    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:16:18.018644 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013681    5626 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013684    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013686    5626 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013689    5626 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013693    5626 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013695    5626 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013698    5626 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013701    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013704    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013706    5626 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013709    5626 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013711    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013714    5626 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013717    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013720    5626 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013723    5626 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013725    5626 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013728    5626 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013730    5626 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:16:18.019084 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013733    5626 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013735    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013738    5626 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013741    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.013745    5626 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013835    5626 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013845    5626 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013852    5626 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013856    5626 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013860    5626 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013864    5626 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013868    5626 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013872    5626 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013876    5626 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013879    5626 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013882    5626 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013885    5626 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013890    5626 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013893    5626 flags.go:64] FLAG: --cgroup-root=""
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013896    5626 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013899    5626 flags.go:64] FLAG: --client-ca-file=""
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013902    5626 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013905    5626 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 07:16:18.019554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013909    5626 flags.go:64] FLAG: --cluster-domain=""
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013912    5626 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013915    5626 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013918    5626 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013921    5626 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013925    5626 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013928    5626 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013930    5626 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013934    5626 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013937    5626 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013940    5626 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013943    5626 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013946    5626 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013950    5626 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013955    5626 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013958    5626 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013960    5626 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013963    5626 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013966    5626 flags.go:64] FLAG: --enable-server="true"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013969    5626 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013973    5626 flags.go:64] FLAG: --event-burst="100"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013976    5626 flags.go:64] FLAG: --event-qps="50"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013979    5626 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013982    5626 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 07:16:18.020119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013985    5626 flags.go:64] FLAG: --eviction-hard=""
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013989    5626 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013994    5626 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.013997    5626 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014000    5626 flags.go:64] FLAG: --eviction-soft=""
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014003    5626 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014006    5626 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014009    5626 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014012    5626 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014015    5626 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014018    5626 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014020    5626 flags.go:64] FLAG: --feature-gates=""
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014024    5626 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014027    5626 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014030    5626 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014034    5626 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014037    5626 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014040    5626 flags.go:64] FLAG: --help="false"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014042    5626 flags.go:64] FLAG: --hostname-override="ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014046    5626 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014049    5626 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014054    5626 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014058    5626 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 07:16:18.020784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014061    5626 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014064    5626 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014067    5626 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014070    5626 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014073    5626 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014076    5626 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014079    5626 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014082    5626 flags.go:64] FLAG: --kube-reserved=""
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014085    5626 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014088    5626 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014091    5626 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014095    5626 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014098    5626 flags.go:64] FLAG: --lock-file=""
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014101    5626 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014104    5626 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014107    5626 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014112    5626 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014115    5626 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014119    5626 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014122    5626 flags.go:64] FLAG: --logging-format="text"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014125    5626 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014128    5626 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014131    5626 flags.go:64] FLAG: --manifest-url=""
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014134    5626 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014138    5626 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 07:16:18.021838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014141    5626 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014145    5626 flags.go:64] FLAG: --max-pods="110"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014148    5626 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014151    5626 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014154    5626 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014159    5626 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014161    5626 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014164    5626 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014167    5626 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014174    5626 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014177    5626 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014180    5626 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014183    5626 flags.go:64] FLAG: --pod-cidr=""
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014186    5626 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014189    5626 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014192    5626 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014195    5626 flags.go:64] FLAG: --port="10250"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014198    5626 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014202    5626 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-08cefc27ae57d773b"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014205    5626 flags.go:64] FLAG: --qos-reserved=""
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014208    5626 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014211    5626 flags.go:64] FLAG: --register-node="true"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014214    5626 flags.go:64] FLAG: --register-with-taints=""
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014218    5626 flags.go:64] FLAG: --registry-burst="10"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014220    5626 flags.go:64] FLAG: --registry-qps="5"
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014223    5626 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 07:16:18.022461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014227    5626 flags.go:64] FLAG: --reserved-memory=""
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014231    5626 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014234    5626 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014237    5626 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014240    5626 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014243    5626 flags.go:64] FLAG: --runonce="false"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014246    5626 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014249    5626 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014252    5626 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014255    5626 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014258    5626 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014262    5626 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014265    5626 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014268    5626 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014271    5626 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014274    5626 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014277    5626 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014280    5626 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014283    5626 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014286    5626 flags.go:64] FLAG: --system-cgroups=""
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014289    5626 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014293    5626 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014296    5626 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014299    5626 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014304    5626 flags.go:64] FLAG: --tls-min-version=""
Nov 04 07:16:18.023167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014307    5626 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014310    5626 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014313    5626 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014316    5626 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014319    5626 flags.go:64] FLAG: --v="2"
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014323    5626 flags.go:64] FLAG: --version="false"
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014327    5626 flags.go:64] FLAG: --vmodule=""
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014331    5626 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014335    5626 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014481    5626 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014485    5626 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014488    5626 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014490    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014493    5626 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014496    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014499    5626 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014502    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014504    5626 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014509    5626 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014511    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014514    5626 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:16:18.023847 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014517    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014519    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014522    5626 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014524    5626 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014527    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014530    5626 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014534    5626 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014537    5626 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014540    5626 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014542    5626 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014547    5626 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014551    5626 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014555    5626 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014559    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014561    5626 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014564    5626 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014567    5626 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014569    5626 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014572    5626 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:16:18.024488 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014574    5626 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014577    5626 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014580    5626 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014583    5626 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014585    5626 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014588    5626 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014590    5626 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014593    5626 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014595    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014598    5626 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014602    5626 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014605    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014607    5626 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014610    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014612    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014615    5626 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014617    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014620    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014622    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014625    5626 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:16:18.025076 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014628    5626 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014630    5626 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014633    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014636    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014639    5626 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014641    5626 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014644    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014646    5626 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014649    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014652    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014655    5626 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014657    5626 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014660    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014662    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014665    5626 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014668    5626 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014671    5626 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014673    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014676    5626 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:16:18.025734 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014678    5626 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014681    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014683    5626 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014687    5626 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014690    5626 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014693    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014696    5626 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014699    5626 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014702    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014704    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014707    5626 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014709    5626 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014712    5626 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014714    5626 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014717    5626 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014719    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014723    5626 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014726    5626 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014728    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014731    5626 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:16:18.026219 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014733    5626 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014736    5626 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014739    5626 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014742    5626 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014745    5626 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014748    5626 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014750    5626 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014753    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014756    5626 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014758    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014761    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014764    5626 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014766    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014769    5626 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014771    5626 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014775    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:16:18.026720 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.014800    5626 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014833    5626 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014837    5626 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014840    5626 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014844    5626 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014846    5626 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014849    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014852    5626 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014854    5626 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014857    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014860    5626 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014862    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014865    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014869    5626 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014871    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014874    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014877    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014879    5626 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014882    5626 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:16:18.027200 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014884    5626 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014887    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014889    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014892    5626 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014894    5626 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014897    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014900    5626 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014903    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014906    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014909    5626 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014911    5626 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014914    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014917    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014920    5626 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014922    5626 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014925    5626 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014928    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014930    5626 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014933    5626 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:16:18.027856 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014935    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014939    5626 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014942    5626 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014944    5626 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014947    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014950    5626 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014952    5626 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014958    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014961    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014964    5626 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014966    5626 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014969    5626 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014972    5626 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014974    5626 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014977    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014980    5626 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014982    5626 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014985    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014987    5626 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014990    5626 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:16:18.028455 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014993    5626 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014996    5626 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.014999    5626 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015001    5626 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015005    5626 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015009    5626 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015012    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015014    5626 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015017    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015019    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015022    5626 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015025    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015027    5626 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015030    5626 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015032    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015035    5626 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015037    5626 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015040    5626 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015042    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015046    5626 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:16:18.028951 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015049    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015051    5626 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015054    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015056    5626 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015059    5626 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015062    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015064    5626 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015067    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015069    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015072    5626 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015075    5626 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015077    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015080    5626 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015084    5626 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015087    5626 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015090    5626 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015093    5626 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015097    5626 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015102    5626 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:16:18.029448 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015105    5626 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015108    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015111    5626 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015114    5626 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015118    5626 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015120    5626 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015123    5626 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015126    5626 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015128    5626 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.015131    5626 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.018855    5626 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.018871    5626 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.018888    5626 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.018893    5626 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018949    5626 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018954    5626 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018957    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018961    5626 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018963    5626 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018967    5626 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:16:18.029893 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018969    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018972    5626 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018975    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018977    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018980    5626 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018983    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018986    5626 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018989    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018992    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018995    5626 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.018997    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019000    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019003    5626 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019006    5626 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019008    5626 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019011    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019014    5626 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019018    5626 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019021    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019025    5626 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:16:18.030351 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019027    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019030    5626 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019033    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019035    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019038    5626 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019041    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019043    5626 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019046    5626 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019049    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019051    5626 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019054    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019056    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019059    5626 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019061    5626 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019064    5626 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019067    5626 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019070    5626 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019072    5626 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019075    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:16:18.030837 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019078    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019081    5626 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019083    5626 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019086    5626 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019088    5626 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019091    5626 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019094    5626 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019096    5626 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019099    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019101    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019104    5626 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019106    5626 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019109    5626 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019111    5626 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019114    5626 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019116    5626 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019120    5626 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019123    5626 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019126    5626 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019128    5626 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019131    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:16:18.031284 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019133    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019136    5626 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019138    5626 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019141    5626 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019143    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019146    5626 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019148    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019151    5626 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019153    5626 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019156    5626 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019158    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019161    5626 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019164    5626 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019167    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019170    5626 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019173    5626 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019175    5626 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019178    5626 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019181    5626 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:16:18.031879 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019183    5626 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019186    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019189    5626 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019191    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019194    5626 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019196    5626 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019199    5626 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019203    5626 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019206    5626 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019209    5626 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019211    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019214    5626 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019216    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019219    5626 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019221    5626 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019224    5626 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019226    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019229    5626 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019231    5626 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:16:18.032342 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019234    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019236    5626 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.019261    5626 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019416    5626 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019422    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019425    5626 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019427    5626 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019431    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019434    5626 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019436    5626 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019438    5626 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019441    5626 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019444    5626 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019446    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019449    5626 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019451    5626 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:16:18.032864 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019454    5626 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019456    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019459    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019461    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019464    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019466    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019469    5626 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019471    5626 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019474    5626 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019476    5626 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019479    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019481    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019484    5626 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019487    5626 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019489    5626 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019492    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019494    5626 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019497    5626 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019499    5626 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:16:18.033257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019502    5626 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019504    5626 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019507    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019509    5626 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019512    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019517    5626 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019520    5626 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019522    5626 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019525    5626 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019527    5626 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019530    5626 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019532    5626 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019535    5626 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019538    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019540    5626 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019543    5626 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019545    5626 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019547    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019550    5626 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019553    5626 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:16:18.033745 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019555    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019558    5626 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019561    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019564    5626 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019566    5626 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019569    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019571    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019574    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019577    5626 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019581    5626 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019583    5626 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019586    5626 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019588    5626 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019591    5626 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019593    5626 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019596    5626 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019599    5626 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019601    5626 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019604    5626 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:16:18.034218 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019607    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019609    5626 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019612    5626 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019614    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019616    5626 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019619    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019621    5626 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019624    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019626    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019629    5626 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019631    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019633    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019636    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019638    5626 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019641    5626 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019643    5626 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019646    5626 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019648    5626 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019651    5626 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019653    5626 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:16:18.034667 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019655    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019658    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019660    5626 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019663    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019666    5626 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019668    5626 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019670    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019673    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019675    5626 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019678    5626 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019680    5626 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019682    5626 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019685    5626 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019689    5626 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019693    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:16:18.035134 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.019715    5626 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019751    5626 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019755    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019758    5626 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019761    5626 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019764    5626 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019766    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019769    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019771    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019774    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019776    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019785    5626 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019788    5626 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019790    5626 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019793    5626 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019795    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019798    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019803    5626 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019808    5626 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 07:16:18.035495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019810    5626 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019813    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019835    5626 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019838    5626 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019841    5626 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019843    5626 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019846    5626 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019849    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019851    5626 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019854    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019856    5626 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019860    5626 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019862    5626 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019865    5626 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019868    5626 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019870    5626 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019873    5626 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019875    5626 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019878    5626 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019880    5626 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 07:16:18.035966 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019883    5626 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019886    5626 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019888    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019891    5626 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019893    5626 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019896    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019899    5626 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019901    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019904    5626 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019906    5626 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019910    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019914    5626 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019917    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019921    5626 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019924    5626 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019927    5626 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019930    5626 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019933    5626 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019935    5626 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019938    5626 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 07:16:18.036504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019941    5626 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019944    5626 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019947    5626 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019950    5626 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019953    5626 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019956    5626 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019958    5626 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019961    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019963    5626 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019966    5626 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019968    5626 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019971    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019973    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019976    5626 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019979    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019981    5626 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019984    5626 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019986    5626 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019989    5626 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019991    5626 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 07:16:18.037057 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019994    5626 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.019996    5626 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020000    5626 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020004    5626 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020007    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020009    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020012    5626 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020014    5626 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020017    5626 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020020    5626 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020022    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020025    5626 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020027    5626 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020030    5626 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020033    5626 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020035    5626 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020038    5626 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020042    5626 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 07:16:18.037610 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020046    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020049    5626 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020052    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020055    5626 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020057    5626 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020060    5626 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020063    5626 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020066    5626 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020069    5626 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:18.020071    5626 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.020118    5626 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.022187    5626 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.022239    5626 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.022464    5626 server.go:1014] "Starting client certificate rotation"
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.022610    5626 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.022661    5626 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 07:10:07 +0000 UTC" deadline="2028-06-20 05:12:52.104511737 +0000 UTC"
Nov 04 07:16:18.038305 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.022699    5626 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14253h56m34.081815779s"
Nov 04 07:16:18.038793 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.022912    5626 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 07:16:18.038793 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.024121    5626 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 07:16:18.038793 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.025247    5626 log.go:25] "Validated CRI v1 runtime API"
Nov 04 07:16:18.038793 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.027096    5626 log.go:25] "Validated CRI v1 image API"
Nov 04 07:16:18.038793 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.027859    5626 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 07:16:18.038793 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.029924    5626 fs.go:135] Filesystem UUIDs: map[0c430ad2-fe0c-4b88-a9aa-e39b84f9f905:/dev/nvme0n1p4 1ee09380-3608-40f8-b4dd-1325bc031102:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Nov 04 07:16:18.039016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.029944    5626 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/087e7de43b69fa7fe5b19fa18f72cb7a1d2a6e1e49359abbd50084cdf2ced98c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/087e7de43b69fa7fe5b19fa18f72cb7a1d2a6e1e49359abbd50084cdf2ced98c/userdata/shm major:0 minor:49 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1d291fe1b129b22f99b2ac41e75ffa3d1eaa271e6e081c6723b685251a1c4859/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1d291fe1b129b22f99b2ac41e75ffa3d1eaa271e6e081c6723b685251a1c4859/userdata/shm major:0 minor:239 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3f8e8acf59d0d1e1415ce37b581e593e8ad65763b87c6f7529bd85ce14edbe7e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3f8e8acf59d0d1e1415ce37b581e593e8ad65763b87c6f7529bd85ce14edbe7e/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/40ade68d52d8cb0c72b6bd3b6e4d90ef5dfda552f677870bfa90d8716044f9c1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/40ade68d52d8cb0c72b6bd3b6e4d90ef5dfda552f677870bfa90d8716044f9c1/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/44ee99dbda153808c7164262887164eaaa6fbc4589a4724f11076d13dd0206ad/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/44ee99dbda153808c7164262887164eaaa6fbc4589a4724f11076d13dd0206ad/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5362822ccb25ab840a2cbfd2e4e3e9913d811b7d2d7acd92265f8e42093b7c4e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5362822ccb25ab840a2cbfd2e4e3e9913d811b7d2d7acd92265f8e42093b7c4e/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/56db6e6952766514314b8f6471698a3b60340778e554eda21c9809ab4edb5557/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/56db6e6952766514314b8f6471698a3b60340778e554eda21c9809ab4edb5557/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6d63d36dfd91395a2a336c91682cff45aea0eba596c6c8e305a4a618d84bdb46/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6d63d36dfd91395a2a336c91682cff45aea0eba596c6c8e305a4a618d84bdb46/userdata/shm major:0 minor:382 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/750a8d71e0a350c6489ceec8d75f40a7e652c9ebeca3a5ddb74a05a355aefad3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/750a8d71e0a350c6489ceec8d75f40a7e652c9ebeca3a5ddb74a05a355aefad3/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7d9052f080f1cadd5cbc1adca8b3b37ce84b4b7a6177a3026970461146e91070/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7d9052f080f1cadd5cbc1adca8b3b37ce84b4b7a6177a3026970461146e91070/userdata/shm major:0 minor:241 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ac05179d7224574da2fd7d92069268b9575f4ffe0323bffcac63f1df0213cb02/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ac05179d7224574da2fd7d92069268b9575f4ffe0323bffcac63f1df0213cb02/userdata/shm major:0 minor:237 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b191bb4676bff161861b6bc9b7f749df12b41429017cd23c1505ade498400946/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b191bb4676bff161861b6bc9b7f749df12b41429017cd23c1505ade498400946/userdata/shm major:0 minor:431 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c1577594e6a15cedeff5be6a89d191f37b2b9095f9c92a1ddc7081cbf6b144ba/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c1577594e6a15cedeff5be6a89d191f37b2b9095f9c92a1ddc7081cbf6b144ba/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c4d85dd29266327e7e78a085190698c87f6059d9338a9b0cfbedc52aeab52545/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c4d85dd29266327e7e78a085190698c87f6059d9338a9b0cfbedc52aeab52545/userdata/shm major:0 minor:435 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d6873c187f2c08e1a814be480c98c18baf108443511067bf4f4546299ccb20a3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d6873c187f2c08e1a814be480c98c18baf108443511067bf4f4546299ccb20a3/userdata/shm major:0 minor:291 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d71980dfca8b151274ef89063a68a58fa1bb2e0cf015d0f14db65e7d2f013042/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d71980dfca8b151274ef89063a68a58fa1bb2e0cf015d0f14db65e7d2f013042/userdata/shm major:0 minor:336 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dbd3e16554021dab0d69cbf5ae8293127e610701a980ff0de8eecf00e2a7cb89/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dbd3e16554021dab0d69cbf5ae8293127e610701a980ff0de8eecf00e2a7cb89/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dbdf490e980d741d51b0cbaf6cd1551606971f6c843afd7fb336e9eeaef7be98/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dbdf490e980d741d51b0cbaf6cd1551606971f6c843afd7fb336e9eeaef7be98/userdata/shm major:0 minor:89 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e2ce534a29576f7fe285a3207595ab17683ee456d4584eb52d3abd8d7e87e374/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e2ce534a29576f7fe285a3207595ab17683ee456d4584eb52d3abd8d7e87e374/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e4179b3bef76fa9449295a47daf08f6d763af821e9e8062124f92049abfce80f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e4179b3bef76fa9449295a47daf08f6d763af821e9e8062124f92049abfce80f/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ebeed33edf2dbe9ce87c4318b7629823a1c51d90cdc7bcfca50af8354d9235a1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ebeed33edf2dbe9ce87c4318b7629823a1c51d90cdc7bcfca50af8354d9235a1/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f5d07436b33e831ddf14cc9ad79f505b3aed34f0b09d534370911f772f0b6817/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f5d07436b33e831ddf14cc9ad79f505b3aed34f0b09d534370911f772f0b6817/userdata/shm major:0 minor:433 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f6efbf2d5da3a0a56cbf1edb25965f87282945eb79bfd9f2664647e8d87d5b52/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f6efbf2d5da3a0a56cbf1edb25965f87282945eb79bfd9f2664647e8d87d5b52/userdata/shm major:0 minor:350 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/03300026-6052-474a-a6b0-53afd4f3f67a/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/03300026-6052-474a-a6b0-53afd4f3f67a/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/03300026-6052-474a-a6b0-53afd4f3f67a/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/03300026-6052-474a-a6b0-53afd4f3f67a/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/03300026-6052-474a-a6b0-53afd4f3f67a/volumes/kubernetes.io~projected/kube-api-access-n4qxf:{mountpoint:/var/lib/kubelet/pods/03300026-6052-474a-a6b0-53afd4f3f67a/volumes/kubernetes.io~projected/kube-api-access-n4qxf major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/07aa084c-c07a-4901-800d-a3ba2a659bd5/volumes/kubernetes.io~projected/kube-api-access-59cl7:{mountpoint:/var/lib/kubelet/pods/07aa084c-c07a-4901-800d-a3ba2a659bd5/volumes/kubernetes.io~projected/kube-api-access-59cl7 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0d74bcca-3fbc-4984-8acf-b53b0f34e30a/volumes/kubernetes.io~projected/kube-api-access-6tvmk:{mountpoint:/var/lib/kubelet/pods/0d74bcca-3fbc-4984-8acf-b53b0f34e30a/volumes/kubernetes.io~projected/kube-api-access-6tvmk major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ef74e4a-dee2-4554-be57-0eda5677ebf2/volumes/kubernetes.io~projected/kube-api-access-g4hwz:{mountpoint:/var/lib/kubelet/pods/0ef74e4a-dee2-4554-be57-0eda5677ebf2/volumes/kubernetes.io~projected/kube-api-access-g4hwz major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ef74e4a-dee2-4554-be57-0eda5677ebf2/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/0ef74e4a-dee2-4554-be57-0eda5677ebf2/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ef74e4a-dee2-4554-be57-0eda5677ebf2/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/0ef74e4a-dee2-4554-be57-0eda5677ebf2/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ef74e4a-dee2-4554-be57-0eda5677ebf2/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/0ef74e4a-dee2-4554-be57-0eda5677ebf2/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13b111e6-49a0-45e4-bb33-a49c5dc4d439/volumes/kubernetes.io~projected/kube-api-access-n4fxt:{mountpoint:/var/lib/kubelet/pods/13b111e6-49a0-45e4-bb33-a49c5dc4d439/volumes/kubernetes.io~projected/kube-api-access-n4fxt major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13b111e6-49a0-45e4-bb33-a49c5dc4d439/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/13b111e6-49a0-45e4-bb33-a49c5dc4d439/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13b111e6-49a0-45e4-bb33-a49c5dc4d439/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/13b111e6-49a0-45e4-bb33-a49c5dc4d439/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6/volumes/kubernetes.io~projected/kube-api-access-5zrpx:{mountpoint:/var/lib/kubelet/pods/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6/volumes/kubernetes.io~projected/kube-api-access-5zrpx major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24419200-a11c-445d-8a5b-d35e6473614f/volumes/kubernetes.io~projected/kube-api-access-2bwzv:{mountpoint:/var/lib/kubelet/pods/24419200-a11c-445d-8a5b-d35e6473614f/volumes/kubernetes.io~projected/kube-api-access-2bwzv major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24419200-a11c-445d-8a5b-d35e6473614f/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/24419200-a11c-445d-8a5b-d35e6473614f/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05/volumes/kubernetes.io~projected/kube-api-access-pbnh9:{mountpoint:/var/lib/kubelet/pods/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05/volumes/kubernetes.io~projected/kube-api-access-pbnh9 major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:343 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3f1d3530-4c03-4633-92b1-9fc2ab954b9f/volumes/kubernetes.io~projected/kube-api-access-xd74j:{mountpoint:/var/lib/kubelet/pods/3f1d3530-4c03-4633-92b1-9fc2ab954b9f/volumes/kubernetes.io~projected/kube-api-access-xd74j major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52e5228a-5720-4bda-9af5-2c1a2d014923/volumes/kubernetes.io~projected/kube-api-access-rtwxm:{mountpoint:/var/lib/kubelet/pods/52e5228a-5720-4bda-9af5-2c1a2d014923/volumes/kubernetes.io~projected/kube-api-access-rtwxm major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52e5228a-5720-4bda-9af5-2c1a2d014923/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/52e5228a-5720-4bda-9af5-2c1a2d014923/volumes/kubernetes.io~secret/metrics-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/542b9c39-a718-4064-9640-5123b2dcb358/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/542b9c39-a718-4064-9640-5123b2dcb358/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/542b9c39-a718-4064-9640-5123b2dcb358/volumes/kubernetes.io~projected/kube-api-access-7rt4f:{mountpoint:/var/lib/kubelet/pods/542b9c39-a718-4064-9640-5123b2dcb358/volumes/kubernetes.io~projected/kube-api-access-7rt4f major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/542b9c39-a718-4064-9640-5123b2dcb358/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/542b9c39-a718-4064-9640-5123b2dcb358/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/723c3b52-1b88-45aa-9ab1-c0a712bf733e/volumes/kubernetes.io~projected/kube-api-access-cqrv7:{mountpoint:/var/lib/kubelet/pods/723c3b52-1b88-45aa-9ab1-c0a712bf733e/volumes/kubernetes.io~projected/kube-api-access-cqrv7 major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/723c3b52-1b88-45aa-9ab1-c0a712bf733e/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/723c3b52-1b88-45aa-9ab1-c0a712bf733e/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~projected/kube-api-access-h52sl:{mountpoint:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~projected/kube-api-access-h52sl major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:279 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:273 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:274 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7a10a31c-6c0e-45e1-9d6f-e1532180e474/volumes/kubernetes.io~projected/kube-api-access-8h7nn:{mountpoint:/var/lib/kubelet/pods/7a10a31c-6c0e-45e1-9d6f-e1532180e474/volumes/kubernetes.io~projected/kube-api-access-8h7nn major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7a10a31c-6c0e-45e1-9d6f-e1532180e474/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/7a10a31c-6c0e-45e1-9d6f-e1532180e474/volumes/kubernetes.io~secret/metrics-certs major:0 minor:429 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab0e4439-974e-420d-be04-e9e86c18d606/volumes/kubernetes.io~projected/kube-api-access-48kkb:{mountpoint:/var/lib/kubelet/pods/ab0e4439-974e-420d-be04-e9e86c18d606/volumes/kubernetes.io~projected/kube-api-access-48kkb major:0 minor:430 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd7ea631-0ca9-4829-8408-8352e1d1bc7f/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/bd7ea631-0ca9-4829-8408-8352e1d1bc7f/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:425 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/caee6966-5544-44a7-8b90-c431ef6b94de/volumes/kubernetes.io~projected/kube-api-access-bpmn2:{mountpoint:/var/lib/kubelet/pods/caee6966-5544-44a7-8b90-c431ef6b94de/volumes/kubernetes.io~projected/kube-api-access-bpmn2 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f95c6e6a-6787-423e-8129-a62feb0a5156/volumes/kubernetes.io~projected/kube-api-access-vcd2d:{mountpoint:/var/lib/kubelet/pods/f95c6e6a-6787-423e-8129-a62feb0a5156/volumes/kubernetes.io~projected/kube-api-access-vcd2d major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volume-subpaths/web-config/config-reloader/2 major:0 minor:371 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:371 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volume-subpaths/web-config/prometheus/15 major:0 minor:371 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~empty-dir/config-out major:0 minor:373 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~projected/kube-api-access-mxgcd:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~projected/kube-api-access-mxgcd major:0 minor:383 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~projected/tls-assets major:0 minor:378 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/config major:0 minor:376 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:372 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:380 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:381 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:375 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:379 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:377 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/web-config major:0 minor:371 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe2e0b0d-3d26-4d4d-90a6-ef499452293b/volumes/kubernetes.io~projected/kube-api-access-q7hnk:{mountpoint:/var/lib/kubelet/pods/fe2e0b0d-3d26-4d4d-90a6-ef499452293b/volumes/kubernetes.io~projected/kube-api-access-q7hnk major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe2e0b0d-3d26-4d4d-90a6-ef499452293b/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/fe2e0b0d-3d26-4d4d-90a6-ef499452293b/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe2e0b0d-3d26-4d4d-90a6-ef499452293b/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/fe2e0b0d-3d26-4d4d-90a6-ef499452293b/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fff3c48c-f8c5-4946-a95e-b5faedb4dab8/volumes/kubernetes.io~projected/kube-api-access-f79f6:{mountpoint:/var/lib/kubelet/pods/fff3c48c-f8c5-4946-a95e-b5faedb4dab8/volumes/kubernetes.io~projected/kube-api-access-f79f6 major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fff3c48c-f8c5-4946-a95e-b5faedb4dab8/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/fff3c48c-f8c5-4946-a95e-b5faedb4dab8/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fff3c48c-f8c5-4946-a95e-b5faedb4dab8/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/fff3c48c-f8c5-4946-a95e-b5faedb4dab8/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:230 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/726fa0d616f8ef6330a5b75dab7385b9a6ecf6c2f8043e96bb996a7bafe31939/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/d54eee7cc870d2fec480bba18d1d0f9ca9f44de40690727529ba278c9f7dcc24/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/7d35b9fbadee7e07f0c44c90723b6983bbd8ad99a9f5cee873b53539c07202eb/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/018db75687e55674c34de396652ab548f1624a2d45e3e09abb2ba869d6ea9430/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/dbf592e35752163f16f146e0e60567c7a428d18d0f453ef41f75d4acf8bfb4c9/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/da85c1f8282abaaabe1d124ae15ae8c8bb43ea9c3a668c43b3c4df4c30b0828c/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/9102be929ca19edd3986923fe755754f3815f408da496e1834e141797e87711b/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/891fd1659d9ba2e8ec1db5473b661d9b501a8592ae3b65fdaf8ef92998d474e6/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/435b599be2adcded04160a5e1a0702c63ed19d5a9b28047bd0272f4e98054ccc/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/61165c98d8fe5fc2ff5cb35cdc1f2fe9ec832a7442c733244178add3de9ef165/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-138:{mountpoint:/var/lib/containers/storage/overlay/fdb5b531eb1c7a4ea884a7686550dda203e3ff60e37a3e18c08e29edce6771b5/merged major:0 minor:138 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/8e42493ac0754ae2912b49c2203d6dd5cb72ab6ac707e332f0ac32a828cb2cb9/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/2ad9c788b1d47c2e2a97132dd0f383dead1552aafe720b6f855501560b56903e/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/eef5410aa673fc1165327bbc19fa25f2d3b7a2a55b289bd34a57965f832c80dc/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-159:{mountpoint:/var/lib/containers/storage/overlay/17579a81c4d27cf914e09bf20d523523bfddfddf1570cf30b82163f01b5b4d40/merged major:0 minor:159 fsType:overlay blockSize:0} overlay_0-164:{mountpoint:/var/lib/containers/storage/overlay/50fc75ce2735430190e2d8bd3ae3a8d0ff8b61a242a331c916ee7eb2bfa11c33/merged major:0 minor:164 fsType:overlay blockSize:0} overlay_0-166:{mountpoint:/var/lib/containers/storage/overlay/4ad401008bfa3f8165e3bc4e6c02f464d2726141a5fe3de1db5a6d9507fecff8/merged major:0 minor:166 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/3197f898864d36a81a5a1e1e41f855d4b4ac05bc97e1cabe6657c41402d261e5/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/29c9459027da862ff5de858a28d82dae881bff977e577a5fe1b88d62d6b47c26/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/b9c1414bf25be081e9ee0914ab16a42ee75e7bcf66e90666181916519e8e68dd/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/ab3524663af6af8d206a6a5b03a4805edf9847e1de2dd3feb66693b9ef1d3f05/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/ec5483de52f4ce03a7ee433e237c51a1067d362508512440d3d6b8192cf76794/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/6c8352897fc38e22496ebca05db5344e2d7dbfef92379810a76366aef6bade62/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/80b3ad50e8e7ab688e958bf9d3fcc30b2b622a77237fd57cbaaecaf1c407150e/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/8896ba721b3859cbe43ff66fc68ac57cf787407d91491771e15c1ac51a1bbcad/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/37cca84ef864f6ad83e3b7e594f608f8cc8761372c1773844379ff0e837e35e2/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-243:{mountpoint:/var/lib/containers/storage/overlay/04ec281e2bb3a2afb0c8b048ec46763373da79b5c93f99b59a543e3a9a6091d2/merged major:0 minor:243 fsType:overlay blockSize:0} overlay_0-245:{mountpoint:/var/lib/containers/storage/overlay/1c651c9c06b29fed3c26fb417a9ac5ea90537b02568635b931417ae47bd4474c/merged major:0 minor:245 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/ae83b1f0592cb2de6669162aefa377a352c21db293d5495d81d7aa491af9025f/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-249:{mountpoint:/var/lib/containers/storage/overlay/31d3237c0ac3b5ed1a1408d582de7d1bfdb7a0fa122ab19c5e124ec87890798e/merged major:0 minor:249 fsType:overlay blockSize:0} overlay_0-251:{mountpoint:/var/lib/containers/storage/overlay/608851f7259a54e6efb0c2b2e4bc3ff2970f43f59693a7b170446a7c88b42cd3/merged major:0 minor:251 fsType:overlay blockSize:0} overlay_0-257:{mountpoint:/var/lib/containers/storage/overlay/c866863f3510f6f9bcee6e637c084b5d4e686f0772861ece75422448efbf09b6/merged major:0 minor:257 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/c37dc10b012030d78ff4015683a6c998d53cd27e5e507571810833214ce9f023/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/adfd5dec968d99d740d042fad8fc59c2f92fe8aafc471cec87010ec5f8203fa4/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-275:{mountpoint:/var/lib/containers/storage/overlay/3ccdaa0313fee9a04e99e90c6312f9d38980de554f8f88f11b50b4f78ffeefc1/merged major:0 minor:275 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/33669519d81e9fcf87d32bc832d6711eb90b8f182e109e682153625201b4a276/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/96a920216866c110f11cb9ccc4d605c1969bb378953d45608381f5955e1ce24b/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/9328aeacc5ad39ad5b45dd8c3802cd5aa6889a85f0314290fd78f372decca425/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-303:{mountpoint:/var/lib/containers/storage/overlay/f9434ea268b7dcf688982e4267e2374e1532244aba07b1e479ce00d94ccc0755/merged major:0 minor:303 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/7641eb9b31c8ec9b9a1cee6d08534db1f53b25940656d1f4c203ae07f1db697f/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/96141ea55b2d15805f3ee3a0e7bc24ed686bc825aa2f9c29e79c957bd2a94346/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-320:{mountpoint:/var/lib/containers/storage/overlay/9053d5ca9ee3c53c67c35043426ca111c6ffad91888f48bcc274bea8cf871af3/merged major:0 minor:320 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/f949bc2cd5976f890685909c55fdd3be0a47ad1c17ece23597f2ab931a7aecb2/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-330:{mountpoint:/var/lib/containers/storage/overlay/edbd597e61d507601f70f93b2f7574e3481a22927e62f68ad35e5be0bf438e8b/merged major:0 minor:330 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/33a57a6c6e3dbb355b594197ff32bbca36338ce452abe1830a5969b965c1c3d8/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/d1bde5fdd29c70aa011f44772f00fe74de54773d2449fce1d0ae1b2129cf214a/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/9adaaa199363e98d60180093dd40120ef00abc34e6d2dd62bde861156513c3b8/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/148c9dce4a8af69e4d6db91cfd7c5742cbd70aae563a2b88df1a1d351a25dfad/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-356:{mountpoint:/var/lib/containers/storage/overlay/834cdafbda734a1dd436a7aa1c1567f1ec335c0fb55e1594029c1c447365707b/merged major:0 minor:356 fsType:overlay blockSize:0} overlay_0-361:{mountpoint:/var/lib/containers/storage/overlay/b45894c4206030e7dba8877685461b5e9c57e5deeb0373b7e26f95bd50768f9e/merged major:0 minor:361 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/b47a7cc6d1b1cae65ec5ec9588fb3041f2c98a021b5a195ae6642eda46b716e4/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-389:{mountpoint:/var/lib/containers/storage/overlay/768285abc7ccb02fe68e591337cf3123c0a7b9143f2d74d548c1aab173995c25/merged major:0 minor:389 fsType:overlay blockSize:0} overlay_0-391:{mountpoint:/var/lib/containers/storage/overlay/ba53ed575b2ddb48bbf7bf636816848908f470df0d73063844bef744c2c727cd/merged major:0 minor:391 fsType:overlay blockSize:0} overlay_0-397:{mountpoint:/var/lib/containers/storage/overlay/0ab0928c62b2357e8c80f9543ea941ee2de1b2cb1b3f2b068f0d2d04ff5a0909/merged major:0 minor:397 fsType:overlay blockSize:0} overlay_0-399:{mountpoint:/var/lib/containers/storage/overlay/49b7a9fb9e660158b24087a3b25fb6b65e3c4c6aa37ee4318958d4a28a8b0f2c/merged major:0 minor:399 fsType:overlay blockSize:0} overlay_0-405:{mountpoint:/var/lib/containers/storage/overlay/b133ef3119e6c21b48ea168a79fce79ddec915de3198301c1de858e21483ec85/merged major:0 minor:405 fsType:overlay blockSize:0} overlay_0-410:{mountpoint:/var/lib/containers/storage/overlay/f7fd200449b70ccc8bbedecb10cc73f1385bd7472d1ce42e23668c489bcdaea3/merged major:0 minor:410 fsType:overlay blockSize:0} overlay_0-415:{mountpoint:/var/lib/containers/storage/overlay/a19f8bb255f466cca877de20233cc829bae97bbd6ff51f17dcf4de8b4b578cc7/merged major:0 minor:415 fsType:overlay blockSize:0} overlay_0-420:{mountpoint:/var/lib/containers/storage/overlay/221f06cbdcb7fe58da524380e796c03247eede427d1ff6d1808db6fea1c3a86f/merged major:0 minor:420 fsType:overlay blockSize:0} overlay_0-437:{mountpoint:/var/lib/containers/storage/overlay/735970f624e6217efc04030c827fb7ff79a10e00d1e52a684b1c28c60d65d731/merged major:0 minor:437 fsType:overlay blockSize:0} overlay_0-439:{mountpoint:/var/lib/containers/storage/overlay/2072489e3c57a043085928ba48ecb1b3378bce302840575e45f68cf483ac206d/merged major:0 minor:439 fsType:overlay blockSize:0} overlay_0-441:{mountpoint:/var/lib/containers/storage/overlay/83cb4a98b014285632e0b7d851cd7f4a3bcedbfbe4ab1f8b46783a2393f1fd28/merged major:0 minor:441 fsType:overlay blockSize:0} overlay_0-443:{mountpoint:/var/lib/containers/storage/overlay/e7fbc56118ded62b69e4d5d5da5049b8276831b99c64f97327102fff1ef57294/merged major:0 minor:443 fsType:overlay blockSize:0} overlay_0-445:{mountpoint:/var/lib/containers/storage/overlay/3a80a7f14e615871a8c914a4b69ce2822deae6c66f2f671a7bf89caede614681/merged major:0 minor:445 fsType:overlay blockSize:0} overlay_0-451:{mountpoint:/var/lib/containers/storage/overlay/4ca8a34da3eeb50f4f3cb22ad3e0b91b32fa6cafa7ef3d849a285608377e5988/merged major:0 minor:451 fsType:overlay blockSize:0} overlay_0-456:{mountpoint:/var/lib/containers/storage/overlay/f3ae561fae8b73f64ee1a8fa42b542ea538df657f78f369eaac9402044b801bc/merged major:0 minor:456 fsType:overlay blockSize:0} overlay_0-50:{mountpoint:/var/lib/containers/storage/overlay/92ba8ad4c1da7fe619b2e464dd06118229aa84b9152ca056e1909b33cf984af0/merged major:0 minor:50 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/7620681ce390f2a0054c80c0176a30eec4dc11ca31cc46f5401ab26808d4e3a3/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/2afdc1e4c08eaf9b737991711995174b75dd2a210e287db0e3941f820e8301ee/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/4eaf2731cf18f6db88e8c11a5b1868b7d3f23ea49fd46e719e5820d71854ba58/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/b6852cbdbb94ce61d083636f5fc1b5941c5b9a698a3f46e2b98334ee9e24ec9b/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/a17210b64d810128ea1554627c059b31e26132cb1f655cd83b6a81bde302c128/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/4127ac8c3ed2c3eb9e5fd993f4ebf683691e5ec3da43d537c809abe34da409e3/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/7fb35dde6948a8ea58ccd9f7a0ea84ee5067290db146b43d8b30678bb0fe3490/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/750d7c492cf6a1822d559ae9323275d807aa5b14a8e2ed8c46df824361154607/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/5183c342f8a8eff205dfb73f84a5bc2838b1e3f17650f3439d24a78ee2a59884/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 04 07:16:18.039263 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.037351    5626 manager.go:217] Machine: {Timestamp:2026-11-04 07:16:18.036896687 +0000 UTC m=+0.059111938 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3123917 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28419b3d4ba97a94d78d33038463d0 SystemUUID:ec28419b-3d4b-a97a-94d7-8d33038463d0 BootID:aeb749f4-5bf1-4fcd-814a-e26a36c372ca Filesystems:[{Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-166 DeviceMajor:0 DeviceMinor:166 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/723c3b52-1b88-45aa-9ab1-c0a712bf733e/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fff3c48c-f8c5-4946-a95e-b5faedb4dab8/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:231 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:380 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6d63d36dfd91395a2a336c91682cff45aea0eba596c6c8e305a4a618d84bdb46/userdata/shm DeviceMajor:0 DeviceMinor:382 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:371 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-443 DeviceMajor:0 DeviceMinor:443 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ebeed33edf2dbe9ce87c4318b7629823a1c51d90cdc7bcfca50af8354d9235a1/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-164 DeviceMajor:0 DeviceMinor:164 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52e5228a-5720-4bda-9af5-2c1a2d014923/volumes/kubernetes.io~projected/kube-api-access-rtwxm DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d71980dfca8b151274ef89063a68a58fa1bb2e0cf015d0f14db65e7d2f013042/userdata/shm DeviceMajor:0 DeviceMinor:336 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:378 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-451 DeviceMajor:0 DeviceMinor:451 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:343 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/56db6e6952766514314b8f6471698a3b60340778e554eda21c9809ab4edb5557/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:371 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-397 DeviceMajor:0 DeviceMinor:397 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/750a8d71e0a350c6489ceec8d75f40a7e652c9ebeca3a5ddb74a05a355aefad3/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1d291fe1b129b22f99b2ac41e75ffa3d1eaa271e6e081c6723b685251a1c4859/userdata/shm DeviceMajor:0 DeviceMinor:239 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-303 DeviceMajor:0 DeviceMinor:303 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c1577594e6a15cedeff5be6a89d191f37b2b9095f9c92a1ddc7081cbf6b144ba/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/40ade68d52d8cb0c72b6bd3b6e4d90ef5dfda552f677870bfa90d8716044f9c1/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fff3c48c-f8c5-4946-a95e-b5faedb4dab8/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:230 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:273 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-356 DeviceMajor:0 DeviceMinor:356 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-405 DeviceMajor:0 DeviceMinor:405 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/542b9c39-a718-4064-9640-5123b2dcb358/volumes/kubernetes.io~projected/kube-api-access-7rt4f DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:377 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:381 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/caee6966-5544-44a7-8b90-c431ef6b94de/volumes/kubernetes.io~projected/kube-api-access-bpmn2 DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e4179b3bef76fa9449295a47daf08f6d763af821e9e8062124f92049abfce80f/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7d9052f080f1cadd5cbc1adca8b3b37ce84b4b7a6177a3026970461146e91070/userdata/shm DeviceMajor:0 DeviceMinor:241 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:376 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/03300026-6052-474a-a6b0-53afd4f3f67a/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fe2e0b0d-3d26-4d4d-90a6-ef499452293b/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:233 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:277 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b191bb4676bff161861b6bc9b7f749df12b41429017cd23c1505ade498400946/userdata/shm DeviceMajor:0 DeviceMinor:431 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/087e7de43b69fa7fe5b19fa18f72cb7a1d2a6e1e49359abbd50084cdf2ced98c/userdata/shm DeviceMajor:0 DeviceMinor:49 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/24419200-a11c-445d-8a5b-d35e6473614f/volumes/kubernetes.io~projected/kube-api-access-2bwzv DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-245 DeviceMajor:0 DeviceMinor:245 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0ef74e4a-dee2-4554-be57-0eda5677ebf2/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:325 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f6efbf2d5da3a0a56cbf1edb25965f87282945eb79bfd9f2664647e8d87d5b52/userdata/shm DeviceMajor:0 DeviceMinor:350 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f5d07436b33e831ddf14cc9ad79f505b3aed34f0b09d534370911f772f0b6817/userdata/shm DeviceMajor:0 DeviceMinor:433 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3f8e8acf59d0d1e1415ce37b581e593e8ad65763b87c6f7529bd85ce14edbe7e/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6/volumes/kubernetes.io~projected/kube-api-access-5zrpx DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3f1d3530-4c03-4633-92b1-9fc2ab954b9f/volumes/kubernetes.io~projected/kube-api-access-xd74j DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-243 DeviceMajor:0 DeviceMinor:243 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-257 DeviceMajor:0 DeviceMinor:257 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-275 DeviceMajor:0 DeviceMinor:275 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0ef74e4a-dee2-4554-be57-0eda5677ebf2/volumes/kubernetes.io~projected/kube-api-access-g4hwz DeviceMajor:0 DeviceMinor:327 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-330 DeviceMajor:0 DeviceMinor:330 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:373 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/13b111e6-49a0-45e4-bb33-a49c5dc4d439/volumes/kubernetes.io~projected/kube-api-access-n4fxt DeviceMajor:0 DeviceMinor:236 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ac05179d7224574da2fd7d92069268b9575f4ffe0323bffcac63f1df0213cb02/userdata/shm DeviceMajor:0 DeviceMinor:237 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~projected/kube-api-access-mxgcd DeviceMajor:0 DeviceMinor:383 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-415 DeviceMajor:0 DeviceMinor:415 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dbdf490e980d741d51b0cbaf6cd1551606971f6c843afd7fb336e9eeaef7be98/userdata/shm DeviceMajor:0 DeviceMinor:89 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:371 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c4d85dd29266327e7e78a085190698c87f6059d9338a9b0cfbedc52aeab52545/userdata/shm DeviceMajor:0 DeviceMinor:435 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/03300026-6052-474a-a6b0-53afd4f3f67a/volumes/kubernetes.io~projected/kube-api-access-n4qxf DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-50 DeviceMajor:0 DeviceMinor:50 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~projected/kube-api-access-h52sl DeviceMajor:0 DeviceMinor:288 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f95c6e6a-6787-423e-8129-a62feb0a5156/volumes/kubernetes.io~projected/kube-api-access-vcd2d DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0d74bcca-3fbc-4984-8acf-b53b0f34e30a/volumes/kubernetes.io~projected/kube-api-access-6tvmk DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-249 DeviceMajor:0 DeviceMinor:249 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-445 DeviceMajor:0 DeviceMinor:445 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/542b9c39-a718-4064-9640-5123b2dcb358/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-251 DeviceMajor:0 DeviceMinor:251 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:287 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-441 DeviceMajor:0 DeviceMinor:441 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5362822ccb25ab840a2cbfd2e4e3e9913d811b7d2d7acd92265f8e42093b7c4e/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52e5228a-5720-4bda-9af5-2c1a2d014923/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:374 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dbd3e16554021dab0d69cbf5ae8293127e610701a980ff0de8eecf00e2a7cb89/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:274 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-320 DeviceMajor:0 DeviceMinor:320 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0ef74e4a-dee2-4554-be57-0eda5677ebf2/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:326 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd7ea631-0ca9-4829-8408-8352e1d1bc7f/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:425 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/13b111e6-49a0-45e4-bb33-a49c5dc4d439/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0ef74e4a-dee2-4554-be57-0eda5677ebf2/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:319 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:372 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-391 DeviceMajor:0 DeviceMinor:391 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/07aa084c-c07a-4901-800d-a3ba2a659bd5/volumes/kubernetes.io~projected/kube-api-access-59cl7 DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:375 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-159 DeviceMajor:0 DeviceMinor:159 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/13b111e6-49a0-45e4-bb33-a49c5dc4d439/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:229 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fe2e0b0d-3d26-4d4d-90a6-ef499452293b/volumes/kubernetes.io~projected/kube-api-access-q7hnk DeviceMajor:0 DeviceMinor:234 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:278 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-389 DeviceMajor:0 DeviceMinor:389 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/542b9c39-a718-4064-9640-5123b2dcb358/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77e0a9a6-4d92-4f21-8ecf-7b791b33a730/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:279 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-361 DeviceMajor:0 DeviceMinor:361 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-138 DeviceMajor:0 DeviceMinor:138 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-420 DeviceMajor:0 DeviceMinor:420 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7a10a31c-6c0e-45e1-9d6f-e1532180e474/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:429 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/03300026-6052-474a-a6b0-53afd4f3f67a/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d6873c187f2c08e1a814be480c98c18baf108443511067bf4f4546299ccb20a3/userdata/shm DeviceMajor:0 DeviceMinor:291 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-399 DeviceMajor:0 DeviceMinor:399 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-437 DeviceMajor:0 DeviceMinor:437 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7a10a31c-6c0e-45e1-9d6f-e1532180e474/volumes/kubernetes.io~projected/kube-api-access-8h7nn DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:371 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-439 DeviceMajor:0 DeviceMinor:439 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fe2e0b0d-3d26-4d4d-90a6-ef499452293b/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:232 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05/volumes/kubernetes.io~projected/kube-api-access-pbnh9 DeviceMajor:0 DeviceMinor:347 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e2ce534a29576f7fe285a3207595ab17683ee456d4584eb52d3abd8d7e87e374/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/723c3b52-1b88-45aa-9ab1-c0a712bf733e/volumes/kubernetes.io~projected/kube-api-access-cqrv7 DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/44ee99dbda153808c7164262887164eaaa6fbc4589a4724f11076d13dd0206ad/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:379 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ab0e4439-974e-420d-be04-e9e86c18d606/volumes/kubernetes.io~projected/kube-api-access-48kkb DeviceMajor:0 DeviceMinor:430 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-456 DeviceMajor:0 DeviceMinor:456 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/24419200-a11c-445d-8a5b-d35e6473614f/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fff3c48c-f8c5-4946-a95e-b5faedb4dab8/volumes/kubernetes.io~projected/kube-api-access-f79f6 DeviceMajor:0 DeviceMinor:235 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-410 DeviceMajor:0 DeviceMinor:410 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:1d291fe1b129b22 MacAddress:76:51:8a:27:6d:83 Speed:10000 Mtu:8901} {Name:40ade68d52d8cb0 MacAddress:92:3e:91:dc:10:b9 Speed:10000 Mtu:8901} {Name:44ee99dbda15380 MacAddress:32:1c:96:cc:92:23 Speed:10000 Mtu:8901} {Name:6d63d36dfd91395 MacAddress:92:0a:ca:1d:da:93 Speed:10000 Mtu:8901} {Name:ac05179d7224574 MacAddress:ce:d3:fe:00:7f:16 Speed:10000 Mtu:8901} {Name:b191bb4676bff16 MacAddress:ee:ab:3c:e9:39:aa Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:e7:8d:6b:15 Speed:0 Mtu:9001} {Name:br-int MacAddress:9a:46:83:5c:d3:aa Speed:0 Mtu:8901} {Name:c4d85dd29266327 MacAddress:96:d5:22:5d:cc:eb Speed:10000 Mtu:8901} {Name:d6873c187f2c08e MacAddress:a6:cf:f2:64:da:7e Speed:10000 Mtu:8901} {Name:d71980dfca8b151 MacAddress:46:79:43:2a:ef:2d Speed:10000 Mtu:8901} {Name:e4179b3bef76fa9 MacAddress:46:c2:eb:b4:5c:16 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:e7:8d:6b:15 Speed:0 Mtu:9001} {Name:f5d07436b33e831 MacAddress:82:40:d4:97:66:6f Speed:10000 Mtu:8901} {Name:f6efbf2d5da3a0a MacAddress:9e:d7:dd:c8:33:14 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:ce:54:e4:ef:aa:c9 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:d6:3b:70:36:ee:21 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 07:16:18.039263 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.038460    5626 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 07:16:18.039263 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.038520    5626 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 07:16:18.039263 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039136    5626 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 07:16:18.039537 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039163    5626 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-197.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 07:16:18.039537 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039281    5626 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 07:16:18.039537 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039289    5626 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 07:16:18.039537 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039298    5626 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 07:16:18.039537 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039316    5626 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 07:16:18.039537 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039411    5626 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 07:16:18.039692 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039561    5626 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 07:16:18.039692 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039667    5626 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 07:16:18.039740 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039700    5626 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 07:16:18.039740 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039712    5626 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 07:16:18.039740 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039721    5626 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 07:16:18.039740 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039730    5626 kubelet.go:405] "Adding apiserver pod source"
Nov 04 07:16:18.039740 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.039738    5626 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 07:16:18.040465 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.040452    5626 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 07:16:18.040465 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.040465    5626 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 07:16:18.040912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.040896    5626 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 07:16:18.043501 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043487    5626 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 07:16:18.043501 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043501    5626 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 07:16:18.043601 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043596    5626 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 07:16:18.043776 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043765    5626 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 07:16:18.043813 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043801    5626 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 07:16:18.043919 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043910    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 07:16:18.043957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043924    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 07:16:18.043957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043931    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 07:16:18.043957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043936    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 07:16:18.043957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043942    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 07:16:18.043957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043952    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 07:16:18.043957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043958    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 07:16:18.044193 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043964    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 07:16:18.044193 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043971    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 07:16:18.044193 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043976    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 07:16:18.044193 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043989    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 07:16:18.044193 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.043999    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 07:16:18.044193 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.044023    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 07:16:18.044193 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.044030    5626 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 07:16:18.047485 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.047470    5626 server.go:1263] "Started kubelet"
Nov 04 07:16:18.047576 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.047541    5626 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 07:16:18.047624 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.047570    5626 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 07:16:18.047671 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.047633    5626 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 07:16:18.048013 ip-10-0-142-197 systemd[1]: Started Kubernetes Kubelet.
Nov 04 07:16:18.049192 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.049175    5626 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 07:16:18.052859 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.052836    5626 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 07:16:18.053354 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.053340    5626 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 07:16:18.054694 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.053568    5626 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 07:16:18.054694 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.053583    5626 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 07:16:18.054694 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.053689    5626 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 07:16:18.054694 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.054011    5626 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 07:16:18.054694 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.054067    5626 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 07:10:08 +0000 UTC" deadline="2028-06-09 15:53:07.70262945 +0000 UTC"
Nov 04 07:16:18.054694 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.054098    5626 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14000h36m49.648535009s"
Nov 04 07:16:18.054694 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:18.054393    5626 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-197.us-east-2.compute.internal\" not found"
Nov 04 07:16:18.055673 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.055649    5626 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 07:16:18.055673 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.055670    5626 factory.go:55] Registering systemd factory
Nov 04 07:16:18.056016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.055680    5626 factory.go:223] Registration of the systemd container factory successfully
Nov 04 07:16:18.056197 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056173    5626 factory.go:153] Registering CRI-O factory
Nov 04 07:16:18.056197 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056198    5626 factory.go:223] Registration of the crio container factory successfully
Nov 04 07:16:18.056343 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056216    5626 factory.go:103] Registering Raw factory
Nov 04 07:16:18.056343 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056230    5626 manager.go:1199] Started watching for new ooms in manager
Nov 04 07:16:18.056595 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056577    5626 manager.go:319] Starting recovery of all containers
Nov 04 07:16:18.056707 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056611    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" volumeName="kubernetes.io/projected/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-kube-api-access-6tvmk" seLinuxMountContext=""
Nov 04 07:16:18.056707 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056646    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="542b9c39-a718-4064-9640-5123b2dcb358" volumeName="kubernetes.io/projected/542b9c39-a718-4064-9640-5123b2dcb358-kube-api-access-7rt4f" seLinuxMountContext=""
Nov 04 07:16:18.056707 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056661    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="caee6966-5544-44a7-8b90-c431ef6b94de" volumeName="kubernetes.io/configmap/caee6966-5544-44a7-8b90-c431ef6b94de-cni-binary-copy" seLinuxMountContext=""
Nov 04 07:16:18.056707 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056674    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="caee6966-5544-44a7-8b90-c431ef6b94de" volumeName="kubernetes.io/projected/caee6966-5544-44a7-8b90-c431ef6b94de-kube-api-access-bpmn2" seLinuxMountContext=""
Nov 04 07:16:18.056707 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056686    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="07aa084c-c07a-4901-800d-a3ba2a659bd5" volumeName="kubernetes.io/projected/07aa084c-c07a-4901-800d-a3ba2a659bd5-kube-api-access-59cl7" seLinuxMountContext=""
Nov 04 07:16:18.056707 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056701    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77e0a9a6-4d92-4f21-8ecf-7b791b33a730" volumeName="kubernetes.io/projected/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-kube-api-access-h52sl" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056712    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-grpc-tls" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056763    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77e0a9a6-4d92-4f21-8ecf-7b791b33a730" volumeName="kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056777    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ef74e4a-dee2-4554-be57-0eda5677ebf2" volumeName="kubernetes.io/configmap/0ef74e4a-dee2-4554-be57-0eda5677ebf2-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056790    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ef74e4a-dee2-4554-be57-0eda5677ebf2" volumeName="kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-client-ca-bundle" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056804    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13b111e6-49a0-45e4-bb33-a49c5dc4d439" volumeName="kubernetes.io/configmap/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056818    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2904aa1a-7fd1-43c5-922a-2a0a85e2ab05" volumeName="kubernetes.io/projected/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05-kube-api-access-pbnh9" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056830    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52e5228a-5720-4bda-9af5-2c1a2d014923" volumeName="kubernetes.io/projected/52e5228a-5720-4bda-9af5-2c1a2d014923-kube-api-access-rtwxm" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056841    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056857    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056880    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" volumeName="kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cni-binary-copy" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056892    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ef74e4a-dee2-4554-be57-0eda5677ebf2" volumeName="kubernetes.io/projected/0ef74e4a-dee2-4554-be57-0eda5677ebf2-kube-api-access-g4hwz" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056907    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056918    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fff3c48c-f8c5-4946-a95e-b5faedb4dab8" volumeName="kubernetes.io/configmap/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-metrics-client-ca" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056930    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77e0a9a6-4d92-4f21-8ecf-7b791b33a730" volumeName="kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056941    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03300026-6052-474a-a6b0-53afd4f3f67a" volumeName="kubernetes.io/empty-dir/03300026-6052-474a-a6b0-53afd4f3f67a-tmp" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056954    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03300026-6052-474a-a6b0-53afd4f3f67a" volumeName="kubernetes.io/empty-dir/03300026-6052-474a-a6b0-53afd4f3f67a-etc-tuned" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056965    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" volumeName="kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056979    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13b111e6-49a0-45e4-bb33-a49c5dc4d439" volumeName="kubernetes.io/projected/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-api-access-n4fxt" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.056990    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52e5228a-5720-4bda-9af5-2c1a2d014923" volumeName="kubernetes.io/configmap/52e5228a-5720-4bda-9af5-2c1a2d014923-config-volume" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057003    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52e5228a-5720-4bda-9af5-2c1a2d014923" volumeName="kubernetes.io/empty-dir/52e5228a-5720-4bda-9af5-2c1a2d014923-tmp-dir" seLinuxMountContext=""
Nov 04 07:16:18.057015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057015    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="723c3b52-1b88-45aa-9ab1-c0a712bf733e" volumeName="kubernetes.io/projected/723c3b52-1b88-45aa-9ab1-c0a712bf733e-kube-api-access-cqrv7" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057048    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77e0a9a6-4d92-4f21-8ecf-7b791b33a730" volumeName="kubernetes.io/empty-dir/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-tmp" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057063    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52e5228a-5720-4bda-9af5-2c1a2d014923" volumeName="kubernetes.io/secret/52e5228a-5720-4bda-9af5-2c1a2d014923-metrics-tls" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057079    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="723c3b52-1b88-45aa-9ab1-c0a712bf733e" volumeName="kubernetes.io/secret/723c3b52-1b88-45aa-9ab1-c0a712bf733e-insights-runtime-extractor-tls" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057091    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f95c6e6a-6787-423e-8129-a62feb0a5156" volumeName="kubernetes.io/empty-dir/f95c6e6a-6787-423e-8129-a62feb0a5156-tmp-dir" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057106    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057120    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-db" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057132    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-kube-api-access-mxgcd" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057144    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-metrics-client-certs" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057157    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe2e0b0d-3d26-4d4d-90a6-ef499452293b" volumeName="kubernetes.io/secret/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057168    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ef74e4a-dee2-4554-be57-0eda5677ebf2" volumeName="kubernetes.io/configmap/0ef74e4a-dee2-4554-be57-0eda5677ebf2-metrics-server-audit-profiles" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057179    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13da5ce6-d81c-49c2-b92b-ffb77d6a2e38" volumeName="kubernetes.io/secret/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38-agent-certs" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057191    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="542b9c39-a718-4064-9640-5123b2dcb358" volumeName="kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057202    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="723c3b52-1b88-45aa-9ab1-c0a712bf733e" volumeName="kubernetes.io/empty-dir/723c3b52-1b88-45aa-9ab1-c0a712bf733e-data-volume" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057218    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-tls" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057233    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fff3c48c-f8c5-4946-a95e-b5faedb4dab8" volumeName="kubernetes.io/configmap/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057244    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13b111e6-49a0-45e4-bb33-a49c5dc4d439" volumeName="kubernetes.io/secret/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057256    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f1d3530-4c03-4633-92b1-9fc2ab954b9f" volumeName="kubernetes.io/configmap/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-serviceca" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057270    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="723c3b52-1b88-45aa-9ab1-c0a712bf733e" volumeName="kubernetes.io/configmap/723c3b52-1b88-45aa-9ab1-c0a712bf733e-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057282    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057292    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fff3c48c-f8c5-4946-a95e-b5faedb4dab8" volumeName="kubernetes.io/empty-dir/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-textfile" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057307    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f1d3530-4c03-4633-92b1-9fc2ab954b9f" volumeName="kubernetes.io/projected/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-kube-api-access-xd74j" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057319    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="542b9c39-a718-4064-9640-5123b2dcb358" volumeName="kubernetes.io/secret/542b9c39-a718-4064-9640-5123b2dcb358-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057344    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7a10a31c-6c0e-45e1-9d6f-e1532180e474" volumeName="kubernetes.io/projected/7a10a31c-6c0e-45e1-9d6f-e1532180e474-kube-api-access-8h7nn" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057356    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="caee6966-5544-44a7-8b90-c431ef6b94de" volumeName="kubernetes.io/configmap/caee6966-5544-44a7-8b90-c431ef6b94de-multus-daemon-config" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057382    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f95c6e6a-6787-423e-8129-a62feb0a5156" volumeName="kubernetes.io/projected/f95c6e6a-6787-423e-8129-a62feb0a5156-kube-api-access-vcd2d" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057395    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config-out" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057407    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-kube-rbac-proxy" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057420    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe2e0b0d-3d26-4d4d-90a6-ef499452293b" volumeName="kubernetes.io/projected/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-kube-api-access-q7hnk" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057431    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24419200-a11c-445d-8a5b-d35e6473614f" volumeName="kubernetes.io/secret/24419200-a11c-445d-8a5b-d35e6473614f-cert" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057453    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77e0a9a6-4d92-4f21-8ecf-7b791b33a730" volumeName="kubernetes.io/configmap/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-metrics-client-ca" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057465    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77e0a9a6-4d92-4f21-8ecf-7b791b33a730" volumeName="kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-grpc-tls" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057476    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-web-config" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057490    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057503    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe2e0b0d-3d26-4d4d-90a6-ef499452293b" volumeName="kubernetes.io/secret/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-openshift-state-metrics-tls" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057516    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fff3c48c-f8c5-4946-a95e-b5faedb4dab8" volumeName="kubernetes.io/secret/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057529    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24419200-a11c-445d-8a5b-d35e6473614f" volumeName="kubernetes.io/projected/24419200-a11c-445d-8a5b-d35e6473614f-kube-api-access-2bwzv" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057541    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-thanos-prometheus-http-client-file" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057553    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057567    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe2e0b0d-3d26-4d4d-90a6-ef499452293b" volumeName="kubernetes.io/configmap/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-metrics-client-ca" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057578    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fff3c48c-f8c5-4946-a95e-b5faedb4dab8" volumeName="kubernetes.io/projected/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-kube-api-access-f79f6" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057591    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6" volumeName="kubernetes.io/configmap/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-iptables-alerter-script" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057602    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="542b9c39-a718-4064-9640-5123b2dcb358" volumeName="kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-env-overrides" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057614    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77e0a9a6-4d92-4f21-8ecf-7b791b33a730" volumeName="kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057627    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7a10a31c-6c0e-45e1-9d6f-e1532180e474" volumeName="kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057639    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab0e4439-974e-420d-be04-e9e86c18d606" volumeName="kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057655    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057668    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057679    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77e0a9a6-4d92-4f21-8ecf-7b791b33a730" volumeName="kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057691    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" volumeName="kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057702    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13b111e6-49a0-45e4-bb33-a49c5dc4d439" volumeName="kubernetes.io/empty-dir/13b111e6-49a0-45e4-bb33-a49c5dc4d439-volume-directive-shadow" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057714    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13b111e6-49a0-45e4-bb33-a49c5dc4d439" volumeName="kubernetes.io/secret/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-tls" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057727    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6" volumeName="kubernetes.io/projected/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-kube-api-access-5zrpx" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057739    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77e0a9a6-4d92-4f21-8ecf-7b791b33a730" volumeName="kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-tls" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057764    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ef74e4a-dee2-4554-be57-0eda5677ebf2" volumeName="kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-secret-metrics-server-client-certs" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057776    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd7ea631-0ca9-4829-8408-8352e1d1bc7f" volumeName="kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057787    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-metrics-client-ca" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057800    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb89bad1-c644-4d86-b4bf-0827628ac4d3" volumeName="kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-tls-assets" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057811    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03300026-6052-474a-a6b0-53afd4f3f67a" volumeName="kubernetes.io/projected/03300026-6052-474a-a6b0-53afd4f3f67a-kube-api-access-n4qxf" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057823    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ef74e4a-dee2-4554-be57-0eda5677ebf2" volumeName="kubernetes.io/empty-dir/0ef74e4a-dee2-4554-be57-0eda5677ebf2-audit-log" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057837    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ef74e4a-dee2-4554-be57-0eda5677ebf2" volumeName="kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-secret-metrics-server-tls" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057849    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13b111e6-49a0-45e4-bb33-a49c5dc4d439" volumeName="kubernetes.io/configmap/13b111e6-49a0-45e4-bb33-a49c5dc4d439-metrics-client-ca" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057914    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13da5ce6-d81c-49c2-b92b-ffb77d6a2e38" volumeName="kubernetes.io/configmap/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38-konnectivity-ca" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057927    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2904aa1a-7fd1-43c5-922a-2a0a85e2ab05" volumeName="kubernetes.io/secret/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05-monitoring-plugin-cert" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057939    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="542b9c39-a718-4064-9640-5123b2dcb358" volumeName="kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-ovnkube-config" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057951    5626 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fff3c48c-f8c5-4946-a95e-b5faedb4dab8" volumeName="kubernetes.io/secret/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-tls" seLinuxMountContext=""
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057962    5626 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 07:16:18.058047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.057970    5626 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 07:16:18.061250 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.060867    5626 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 07:16:18.063459 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.062908    5626 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 07:16:18.064197 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.063687    5626 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 07:16:18.066669 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.066647    5626 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 07:16:18.083483 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.083451    5626 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 07:16:18.085082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.085056    5626 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 07:16:18.085082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.085079    5626 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 07:16:18.085239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.085116    5626 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 07:16:18.085239 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:18.085157    5626 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 07:16:18.090073 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.090046    5626 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 07:16:18.115540 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.115511    5626 generic.go:356] "Generic (PLEG): container finished" podID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerID="37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7" exitCode=0
Nov 04 07:16:18.131675 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.131644    5626 generic.go:356] "Generic (PLEG): container finished" podID="fff3c48c-f8c5-4946-a95e-b5faedb4dab8" containerID="c9f3ee60a8c8fc79c4ff2340f5e070adec8a7b2578ec4fc277e7841a4f040da5" exitCode=0
Nov 04 07:16:18.133632 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.133608    5626 generic.go:356] "Generic (PLEG): container finished" podID="42efe434931e8b4798eec39818614b9c" containerID="ca176a6ef40dde0757cf8902da6384c12e387b011314867ad762d82e0f3b8c53" exitCode=0
Nov 04 07:16:18.148005 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.147971    5626 generic.go:356] "Generic (PLEG): container finished" podID="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" containerID="62ddb9344a41127f3e6c363a0260ce36b59bbbd5baa3916839099d44f8725204" exitCode=0
Nov 04 07:16:18.148005 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.147992    5626 generic.go:356] "Generic (PLEG): container finished" podID="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" containerID="0037e001f0bd5a3e3bad7a3ca02ad05415743f76161422ba1ed0b15c4564af8d" exitCode=0
Nov 04 07:16:18.148005 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.148001    5626 generic.go:356] "Generic (PLEG): container finished" podID="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" containerID="e8628a47d3435227018be4523640ff82395540e5c1c7b379ec3cdb62512c3c97" exitCode=0
Nov 04 07:16:18.148005 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.148009    5626 generic.go:356] "Generic (PLEG): container finished" podID="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" containerID="f928208fba055cd107539cfcf588fa4c5d7d51ba3a89a236864a1a10ede1d97a" exitCode=0
Nov 04 07:16:18.148267 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.148016    5626 generic.go:356] "Generic (PLEG): container finished" podID="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" containerID="2f59d8a49e023b3eaf2890a42af41459fdbbadb4ca4aad81da75080bce7436a9" exitCode=0
Nov 04 07:16:18.148267 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.148024    5626 generic.go:356] "Generic (PLEG): container finished" podID="0d74bcca-3fbc-4984-8acf-b53b0f34e30a" containerID="bc1c182bd657c8f7a4d2e0e56e1278f318b77fdbdc1f5ae3033207d84355bc09" exitCode=0
Nov 04 07:16:18.166650 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.166597    5626 manager.go:324] Recovery completed
Nov 04 07:16:18.185281 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:18.185242    5626 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 04 07:16:18.189120 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189097    5626 cpu_manager.go:226] "Starting" policy="none"
Nov 04 07:16:18.189120 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189111    5626 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 07:16:18.189246 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189127    5626 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 07:16:18.189277 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189246    5626 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 07:16:18.189277 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189257    5626 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 07:16:18.189348 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189274    5626 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 07:16:18.189348 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189295    5626 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 07:16:18.189348 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189300    5626 policy_none.go:50] "Start"
Nov 04 07:16:18.189348 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189309    5626 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 07:16:18.189348 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189317    5626 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 07:16:18.189510 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189454    5626 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 07:16:18.189510 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189461    5626 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 07:16:18.189510 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.189466    5626 policy_none.go:44] "Start"
Nov 04 07:16:18.193953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.193926    5626 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 07:16:18.196986 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.196968    5626 manager.go:341] "Starting Device Plugin manager"
Nov 04 07:16:18.197642 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:18.196993    5626 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 07:16:18.197684 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.197645    5626 server.go:87] "Starting device plugin registration server"
Nov 04 07:16:18.197907 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.197896    5626 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 07:16:18.197937 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.197911    5626 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 07:16:18.198020 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.198004    5626 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 07:16:18.198128 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.198101    5626 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 07:16:18.198186 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.198137    5626 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 07:16:18.198186 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.198144    5626 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 07:16:18.198278 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.198195    5626 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T07:16:18.198126082Z","UUID":"7231cd97-d60c-4065-8881-cdfced43de08","Handler":null,"Name":"","Endpoint":""}
Nov 04 07:16:18.199527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.199509    5626 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 07:16:18.199619 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.199540    5626 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 07:16:18.200158 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:18.200136    5626 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 07:16:18.298293 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.298265    5626 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 07:16:18.300083 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.300066    5626 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 07:16:18.300134 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.300102    5626 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 07:16:18.300134 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.300116    5626 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-197.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 07:16:18.300188 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.300175    5626 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.317342 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.317319    5626 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.317456 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.317392    5626 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.385623 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.385567    5626 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"]
Nov 04 07:16:18.386278 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.386238    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal" event={"ID":"cec163fb92d8094aef96320b2516e2df","Type":"ContainerStarted","Data":"a3a0be308b7114969659f09b9c9d4eaacbe522e506a105c11bfa37a9b4452040"}
Nov 04 07:16:18.386278 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.386278    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal" event={"ID":"cec163fb92d8094aef96320b2516e2df","Type":"ContainerStarted","Data":"087e7de43b69fa7fe5b19fa18f72cb7a1d2a6e1e49359abbd50084cdf2ced98c"}
Nov 04 07:16:18.386425 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.386328    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal" event={"ID":"42efe434931e8b4798eec39818614b9c","Type":"ContainerStarted","Data":"590331da51ad2be7afca506215625db73809bc52cc293d967fa2c1d9281d844b"}
Nov 04 07:16:18.386425 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.386340    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal" event={"ID":"42efe434931e8b4798eec39818614b9c","Type":"ContainerDied","Data":"ca176a6ef40dde0757cf8902da6384c12e387b011314867ad762d82e0f3b8c53"}
Nov 04 07:16:18.386425 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.386352    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal" event={"ID":"42efe434931e8b4798eec39818614b9c","Type":"ContainerStarted","Data":"5362822ccb25ab840a2cbfd2e4e3e9913d811b7d2d7acd92265f8e42093b7c4e"}
Nov 04 07:16:18.386528 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.386517    5626 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.386595 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.386580    5626 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.400087 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.400070    5626 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 07:16:18.400127 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:18.400111    5626 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.400724 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.400711    5626 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 07:16:18.400768 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:18.400754    5626 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.465059 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.464972    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42efe434931e8b4798eec39818614b9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"42efe434931e8b4798eec39818614b9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.465059 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.465006    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42efe434931e8b4798eec39818614b9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"42efe434931e8b4798eec39818614b9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.465059 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.465041    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cec163fb92d8094aef96320b2516e2df-config\") pod \"kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"cec163fb92d8094aef96320b2516e2df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.565656 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.565623    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42efe434931e8b4798eec39818614b9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"42efe434931e8b4798eec39818614b9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.565656 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.565569    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42efe434931e8b4798eec39818614b9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"42efe434931e8b4798eec39818614b9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.565824 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.565686    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42efe434931e8b4798eec39818614b9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"42efe434931e8b4798eec39818614b9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.565824 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.565706    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cec163fb92d8094aef96320b2516e2df-config\") pod \"kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"cec163fb92d8094aef96320b2516e2df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.565824 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.565731    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cec163fb92d8094aef96320b2516e2df-config\") pod \"kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"cec163fb92d8094aef96320b2516e2df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:18.565824 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:18.565775    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42efe434931e8b4798eec39818614b9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" (UID: \"42efe434931e8b4798eec39818614b9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:19.040292 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.040257    5626 apiserver.go:52] "Watching apiserver"
Nov 04 07:16:19.057663 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.057632    5626 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 07:16:19.058250 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.058230    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-64cb9dd857-hf26d","openshift-multus/multus-dn8mz","openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7","kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal","openshift-monitoring/prometheus-k8s-0","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m","openshift-multus/multus-additional-cni-plugins-k7449","openshift-network-operator/iptables-alerter-qt2pw","openshift-monitoring/metrics-server-76495f7c59-rsjg4","openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc","openshift-multus/network-metrics-daemon-kr5j5","openshift-ovn-kubernetes/ovnkube-node-2gcvm","kube-system/global-pull-secret-syncer-dxhns","openshift-dns/dns-default-2w47v","openshift-image-registry/node-ca-2pxh9","openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd","kube-system/konnectivity-agent-knhlk","openshift-network-diagnostics/network-check-target-smfct","openshift-monitoring/node-exporter-bwnvs","openshift-dns/node-resolver-2zd55","openshift-ingress-canary/ingress-canary-f5wrt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-758f6","openshift-insights/insights-runtime-extractor-5zbxq"]
Nov 04 07:16:19.069377 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.069341    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zw52p\""
Nov 04 07:16:19.069540 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.069338    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tqngx\""
Nov 04 07:16:19.069540 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.069523    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 07:16:19.069668 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.069628    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-slv86\""
Nov 04 07:16:19.071086 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.071061    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 07:16:19.071274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.071125    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 07:16:19.071274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.071132    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 07:16:19.071274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.071146    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 07:16:19.071274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.071178    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l4l9x\""
Nov 04 07:16:19.071274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.071125    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 07:16:19.071274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.071188    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ltv7r\""
Nov 04 07:16:19.071274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.071203    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-hvg8n\""
Nov 04 07:16:19.071274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.071224    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 07:16:19.071274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.071242    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 07:16:19.071274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.071257    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7qf26\""
Nov 04 07:16:19.071604 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.071312    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 07:16:19.072329 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.072309    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 07:16:19.072546 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.072526    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 07:16:19.072628 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.072544    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 07:16:19.072628 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.072538    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 07:16:19.072628 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.072580    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 07:16:19.072628 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.072584    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 07:16:19.072848 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.072828    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 07:16:19.072943 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.072924    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 07:16:19.074265 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074214    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 07:16:19.074265 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074215    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnthh\""
Nov 04 07:16:19.074265 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074247    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gcxz6\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074275    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074282    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074279    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8bggl\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074290    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074326    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074354    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074390    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074404    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074425    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074459    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074475    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074482    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 07:16:19.074493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074484    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 07:16:19.074879 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.074510    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 07:16:19.075042 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.075022    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 07:16:19.075649 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.075638    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 07:16:19.075649 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.075642    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 07:16:19.075794 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.075776    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 07:16:19.075844 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.075808    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 07:16:19.075887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.075858    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 07:16:19.076026 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.076012    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 04 07:16:19.076085 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.076075    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 07:16:19.076135 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.076122    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 07:16:19.076926 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.076912    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 07:16:19.078511 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.078499    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 07:16:19.078935 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.078922    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 04 07:16:19.079953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.079937    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 04 07:16:19.079990 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.079951    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-t6xfx\""
Nov 04 07:16:19.079990 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.079984    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 04 07:16:19.080398 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.080386    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 07:16:19.080441 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.080387    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tjts4\""
Nov 04 07:16:19.080598 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.080563    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-k5j8w\""
Nov 04 07:16:19.080671 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.080572    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 07:16:19.080671 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.080626    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-8kctk\""
Nov 04 07:16:19.080671 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.080644    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9lks7\""
Nov 04 07:16:19.080671 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.080623    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 04 07:16:19.080803 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.080682    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 04 07:16:19.081496 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.081477    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vb7ng\""
Nov 04 07:16:19.081496 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.081492    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 04 07:16:19.081600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.081505    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 04 07:16:19.081600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.081516    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l2xtx\""
Nov 04 07:16:19.081600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.081506    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 04 07:16:19.081600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.081517    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-7rfcz\""
Nov 04 07:16:19.081600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.081547    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 04 07:16:19.081600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.081552    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 07:16:19.081600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.081570    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6pjn4\""
Nov 04 07:16:19.081821 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.081618    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-fcnxr\""
Nov 04 07:16:19.082000 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.081983    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 04 07:16:19.082072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.082022    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 04 07:16:19.082072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.082039    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 04 07:16:19.082072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.082045    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 04 07:16:19.082072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.082054    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 04 07:16:19.082072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.082023    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 04 07:16:19.082211 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.082080    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 04 07:16:19.082392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.082375    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 04 07:16:19.082459 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.082404    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2lqsj\""
Nov 04 07:16:19.082459 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.082408    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 04 07:16:19.083694 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.083683    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 04 07:16:19.083878 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.083863    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 04 07:16:19.084723 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.084712    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7a9sgmd41l2bu\""
Nov 04 07:16:19.087483 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.087462    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 04 07:16:19.087483 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.087468    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 04 07:16:19.087640 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.087498    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 04 07:16:19.087640 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.087532    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2tsas6k3v318c\""
Nov 04 07:16:19.087640 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.087539    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 04 07:16:19.087640 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.087556    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 07:16:19.087776 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.087644    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 04 07:16:19.087776 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.087648    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 07:16:19.087776 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.087706    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-erst5rdl7thnl\""
Nov 04 07:16:19.088409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.088398    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 04 07:16:19.088409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.088402    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jpdvn\""
Nov 04 07:16:19.092688 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.092669    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 07:16:19.093654 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.093635    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 04 07:16:19.155916 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.155890    5626 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:19.156052 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.156035    5626 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:19.157017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.157000    5626 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 07:16:19.169377 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169331    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.169377 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169374    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.169554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169406    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-cni-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.169554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169440    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.169554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169461    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-etc-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.169554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169494    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-env-overrides\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.169728 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169542    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-socket-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.169728 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169657    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:16:19.169728 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169697    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.169728 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169704    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bwzv\" (UniqueName: \"kubernetes.io/projected/24419200-a11c-445d-8a5b-d35e6473614f-kube-api-access-2bwzv\") pod \"ingress-canary-f5wrt\" (UID: \"24419200-a11c-445d-8a5b-d35e6473614f\") " pod="openshift-ingress-canary/ingress-canary-f5wrt"
Nov 04 07:16:19.169728 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169712    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-env-overrides\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.169953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169730    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.169953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169809    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.169953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169873    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a10a31c-6c0e-45e1-9d6f-e1532180e474-metrics-certs\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:16:19.169953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169894    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-ovn\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.170209 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.169987    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.170209 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170007    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-cni-bin\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.170209 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170014    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.170209 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170034    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-registration-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.170209 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170062    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:16:19.170209 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170099    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-kubelet-config\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:16:19.170209 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170119    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-ovnkube-script-lib\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.170209 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170135    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-accelerators-collector-config\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.170209 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170150    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-hostroot\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170210    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03300026-6052-474a-a6b0-53afd4f3f67a-tmp\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170264    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbnh9\" (UniqueName: \"kubernetes.io/projected/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05-kube-api-access-pbnh9\") pod \"monitoring-plugin-77f4f76594-2xqz7\" (UID: \"2904aa1a-7fd1-43c5-922a-2a0a85e2ab05\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170299    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170310    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03300026-6052-474a-a6b0-53afd4f3f67a-tmp\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170322    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170350    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-accelerators-collector-config\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170374    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170376    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48kkb\" (UniqueName: \"kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb\") pod \"network-check-target-smfct\" (UID: \"ab0e4439-974e-420d-be04-e9e86c18d606\") " pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170350    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-ovnkube-script-lib\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170431    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170420    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170475    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f79f6\" (UniqueName: \"kubernetes.io/projected/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-kube-api-access-f79f6\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170519    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-metrics-client-ca\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170534    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170535    5626 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:19.170602    5626 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:19.170641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170556    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-secret-metrics-server-client-certs\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170659    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170705    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-run\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170740    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-cni-multus\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170755    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-metrics-client-ca\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170783    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170814    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-secret-metrics-server-client-certs\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170861    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f95c6e6a-6787-423e-8129-a62feb0a5156-hosts-file\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170867    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170881    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24419200-a11c-445d-8a5b-d35e6473614f-cert\") pod \"ingress-canary-f5wrt\" (UID: \"24419200-a11c-445d-8a5b-d35e6473614f\") " pod="openshift-ingress-canary/ingress-canary-f5wrt"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170914    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-ovnkube-config\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170919    5626 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.170948    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:19.170959    5626 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-197.us-east-2.compute.internal"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171097    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24419200-a11c-445d-8a5b-d35e6473614f-cert\") pod \"ingress-canary-f5wrt\" (UID: \"24419200-a11c-445d-8a5b-d35e6473614f\") " pod="openshift-ingress-canary/ingress-canary-f5wrt"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171101    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171106    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171127    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171180    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-kubelet\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171204    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-run-netns\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171245    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/caee6966-5544-44a7-8b90-c431ef6b94de-cni-binary-copy\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171301    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171307    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/542b9c39-a718-4064-9640-5123b2dcb358-ovnkube-config\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171382    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-multus-certs\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171417    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.171423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171429    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/caee6966-5544-44a7-8b90-c431ef6b94de-cni-binary-copy\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171442    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-tmp\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171385    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171466    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cni-binary-copy\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171497    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-tmp\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171506    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-host\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171479    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171532    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/caee6966-5544-44a7-8b90-c431ef6b94de-multus-daemon-config\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171634    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/13b111e6-49a0-45e4-bb33-a49c5dc4d439-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171641    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171653    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-client-ca-bundle\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171698    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/13b111e6-49a0-45e4-bb33-a49c5dc4d439-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171716    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/caee6966-5544-44a7-8b90-c431ef6b94de-multus-daemon-config\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171724    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171724    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cni-binary-copy\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171760    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tvmk\" (UniqueName: \"kubernetes.io/projected/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-kube-api-access-6tvmk\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171793    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rt4f\" (UniqueName: \"kubernetes.io/projected/542b9c39-a718-4064-9640-5123b2dcb358-kube-api-access-7rt4f\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171817    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysconfig\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171857    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-client-ca-bundle\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171858    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-etc-kubernetes\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171898    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171916    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171937    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-2xqz7\" (UID: \"2904aa1a-7fd1-43c5-922a-2a0a85e2ab05\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171965    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.171988    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172036    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-system-cni-dir\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172080    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f95c6e6a-6787-423e-8129-a62feb0a5156-tmp-dir\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172098    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172136    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-netns\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172150    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-2xqz7\" (UID: \"2904aa1a-7fd1-43c5-922a-2a0a85e2ab05\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172179    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f95c6e6a-6787-423e-8129-a62feb0a5156-tmp-dir\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172166    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h7nn\" (UniqueName: \"kubernetes.io/projected/7a10a31c-6c0e-45e1-9d6f-e1532180e474-kube-api-access-8h7nn\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172204    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172221    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172260    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-dbus\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172303    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpmn2\" (UniqueName: \"kubernetes.io/projected/caee6966-5544-44a7-8b90-c431ef6b94de-kube-api-access-bpmn2\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172315    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-original-pull-secret\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172357    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172392    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172408    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/723c3b52-1b88-45aa-9ab1-c0a712bf733e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172438    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/723c3b52-1b88-45aa-9ab1-c0a712bf733e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172464    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqrv7\" (UniqueName: \"kubernetes.io/projected/723c3b52-1b88-45aa-9ab1-c0a712bf733e-kube-api-access-cqrv7\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172511    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172529    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172538    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172566    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxgcd\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-kube-api-access-mxgcd\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172596    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/723c3b52-1b88-45aa-9ab1-c0a712bf733e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172596    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcd2d\" (UniqueName: \"kubernetes.io/projected/f95c6e6a-6787-423e-8129-a62feb0a5156-kube-api-access-vcd2d\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172635    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-k8s-cni-cncf-io\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172665    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef74e4a-dee2-4554-be57-0eda5677ebf2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172679    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/723c3b52-1b88-45aa-9ab1-c0a712bf733e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172691    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4hwz\" (UniqueName: \"kubernetes.io/projected/0ef74e4a-dee2-4554-be57-0eda5677ebf2-kube-api-access-g4hwz\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172741    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.172727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172768    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-sys\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172785    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172793    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-system-cni-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172800    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172829    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172912    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59cl7\" (UniqueName: \"kubernetes.io/projected/07aa084c-c07a-4901-800d-a3ba2a659bd5-kube-api-access-59cl7\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172892    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef74e4a-dee2-4554-be57-0eda5677ebf2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172950    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-web-config\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.172978    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-tls\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173011    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysctl-conf\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173051    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-socket-dir-parent\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173115    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173133    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtwxm\" (UniqueName: \"kubernetes.io/projected/52e5228a-5720-4bda-9af5-2c1a2d014923-kube-api-access-rtwxm\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173155    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/723c3b52-1b88-45aa-9ab1-c0a712bf733e-data-volume\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173196    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-web-config\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173209    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/723c3b52-1b88-45aa-9ab1-c0a712bf733e-data-volume\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173199    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-grpc-tls\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173222    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-tls\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173250    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0ef74e4a-dee2-4554-be57-0eda5677ebf2-metrics-server-audit-profiles\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173285    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-slash\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173311    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-systemd\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173343    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173393    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52e5228a-5720-4bda-9af5-2c1a2d014923-config-volume\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173422    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13b111e6-49a0-45e4-bb33-a49c5dc4d439-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173445    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03300026-6052-474a-a6b0-53afd4f3f67a-etc-tuned\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173472    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-cni-bin\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173500    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/723c3b52-1b88-45aa-9ab1-c0a712bf733e-crio-socket\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173501    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-grpc-tls\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173528    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173559    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0ef74e4a-dee2-4554-be57-0eda5677ebf2-metrics-server-audit-profiles\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173569    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-iptables-alerter-script\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173568    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03300026-6052-474a-a6b0-53afd4f3f67a-etc-tuned\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173587    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52e5228a-5720-4bda-9af5-2c1a2d014923-config-volume\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173625    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13b111e6-49a0-45e4-bb33-a49c5dc4d439-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173655    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173684    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zrpx\" (UniqueName: \"kubernetes.io/projected/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-kube-api-access-5zrpx\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173710    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-host\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173736    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-textfile\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173763    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-iptables-alerter-script\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173779    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4fxt\" (UniqueName: \"kubernetes.io/projected/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-api-access-n4fxt\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173810    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-secret-metrics-server-tls\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173820    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-textfile\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173837    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173865    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-modprobe-d\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173891    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-kubernetes\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173924    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.173961    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174031    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-systemd-units\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174052    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0ef74e4a-dee2-4554-be57-0eda5677ebf2-secret-metrics-server-tls\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174060    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174081    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-serviceca\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174113    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-wtmp\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174184    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174212    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174219    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-var-lib-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174244    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-log-socket\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174268    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-cni-netd\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174290    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-cnibin\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174314    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38-agent-certs\") pod \"konnectivity-agent-knhlk\" (UID: \"13da5ce6-d81c-49c2-b92b-ffb77d6a2e38\") " pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174377    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7hnk\" (UniqueName: \"kubernetes.io/projected/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-kube-api-access-q7hnk\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174378    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174509    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0ef74e4a-dee2-4554-be57-0eda5677ebf2-audit-log\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174540    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd74j\" (UniqueName: \"kubernetes.io/projected/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-kube-api-access-xd74j\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174568    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-kubelet\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174590    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38-agent-certs\") pod \"konnectivity-agent-knhlk\" (UID: \"13da5ce6-d81c-49c2-b92b-ffb77d6a2e38\") " pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174595    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38-konnectivity-ca\") pod \"konnectivity-agent-knhlk\" (UID: \"13da5ce6-d81c-49c2-b92b-ffb77d6a2e38\") " pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174621    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config-out\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174627    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0ef74e4a-dee2-4554-be57-0eda5677ebf2-audit-log\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174653    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174684    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-root\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174701    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config-out\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174729    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-serviceca\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174780    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysctl-d\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174865    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-systemd\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174913    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13da5ce6-d81c-49c2-b92b-ffb77d6a2e38-konnectivity-ca\") pod \"konnectivity-agent-knhlk\" (UID: \"13da5ce6-d81c-49c2-b92b-ffb77d6a2e38\") " pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174909    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-sys\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.174970    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-lib-modules\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175001    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175014    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:16:19.175017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175061    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175097    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175166    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-node-log\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175196    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-metrics-client-ca\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175210    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175220    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cnibin\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175289    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-sys-fs\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175321    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-tls\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175348    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h52sl\" (UniqueName: \"kubernetes.io/projected/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-kube-api-access-h52sl\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175376    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-metrics-client-ca\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175402    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/542b9c39-a718-4064-9640-5123b2dcb358-ovn-node-metrics-cert\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175412    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175495    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-host-slash\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175523    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175564    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-tls\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175586    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-var-lib-kubelet\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175621    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-device-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175660    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175691    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4qxf\" (UniqueName: \"kubernetes.io/projected/03300026-6052-474a-a6b0-53afd4f3f67a-kube-api-access-n4qxf\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175726    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-conf-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175742    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/542b9c39-a718-4064-9640-5123b2dcb358-ovn-node-metrics-cert\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175754    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52e5228a-5720-4bda-9af5-2c1a2d014923-metrics-tls\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175777    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52e5228a-5720-4bda-9af5-2c1a2d014923-tmp-dir\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175810    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175857    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.175862    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52e5228a-5720-4bda-9af5-2c1a2d014923-tmp-dir\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.176000    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52e5228a-5720-4bda-9af5-2c1a2d014923-metrics-tls\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:16:19.177397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.176028    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.181472 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.181451    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f79f6\" (UniqueName: \"kubernetes.io/projected/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-kube-api-access-f79f6\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.181607 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.181589    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bwzv\" (UniqueName: \"kubernetes.io/projected/24419200-a11c-445d-8a5b-d35e6473614f-kube-api-access-2bwzv\") pod \"ingress-canary-f5wrt\" (UID: \"24419200-a11c-445d-8a5b-d35e6473614f\") " pod="openshift-ingress-canary/ingress-canary-f5wrt"
Nov 04 07:16:19.181668 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.181649    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48kkb\" (UniqueName: \"kubernetes.io/projected/ab0e4439-974e-420d-be04-e9e86c18d606-kube-api-access-48kkb\") pod \"network-check-target-smfct\" (UID: \"ab0e4439-974e-420d-be04-e9e86c18d606\") " pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:16:19.188438 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.188395    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tvmk\" (UniqueName: \"kubernetes.io/projected/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-kube-api-access-6tvmk\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.188609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.188573    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbnh9\" (UniqueName: \"kubernetes.io/projected/2904aa1a-7fd1-43c5-922a-2a0a85e2ab05-kube-api-access-pbnh9\") pod \"monitoring-plugin-77f4f76594-2xqz7\" (UID: \"2904aa1a-7fd1-43c5-922a-2a0a85e2ab05\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:16:19.193054 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.193022    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h7nn\" (UniqueName: \"kubernetes.io/projected/7a10a31c-6c0e-45e1-9d6f-e1532180e474-kube-api-access-8h7nn\") pod \"network-metrics-daemon-kr5j5\" (UID: \"7a10a31c-6c0e-45e1-9d6f-e1532180e474\") " pod="openshift-multus/network-metrics-daemon-kr5j5"
Nov 04 07:16:19.193561 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.193542    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcd2d\" (UniqueName: \"kubernetes.io/projected/f95c6e6a-6787-423e-8129-a62feb0a5156-kube-api-access-vcd2d\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:16:19.193796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.193780    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtwxm\" (UniqueName: \"kubernetes.io/projected/52e5228a-5720-4bda-9af5-2c1a2d014923-kube-api-access-rtwxm\") pod \"dns-default-2w47v\" (UID: \"52e5228a-5720-4bda-9af5-2c1a2d014923\") " pod="openshift-dns/dns-default-2w47v"
Nov 04 07:16:19.194335 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.194319    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpmn2\" (UniqueName: \"kubernetes.io/projected/caee6966-5544-44a7-8b90-c431ef6b94de-kube-api-access-bpmn2\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.195116 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.195094    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4hwz\" (UniqueName: \"kubernetes.io/projected/0ef74e4a-dee2-4554-be57-0eda5677ebf2-kube-api-access-g4hwz\") pod \"metrics-server-76495f7c59-rsjg4\" (UID: \"0ef74e4a-dee2-4554-be57-0eda5677ebf2\") " pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:19.200446 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.200405    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4qxf\" (UniqueName: \"kubernetes.io/projected/03300026-6052-474a-a6b0-53afd4f3f67a-kube-api-access-n4qxf\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.200560 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.200453    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59cl7\" (UniqueName: \"kubernetes.io/projected/07aa084c-c07a-4901-800d-a3ba2a659bd5-kube-api-access-59cl7\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.200560 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.200487    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqrv7\" (UniqueName: \"kubernetes.io/projected/723c3b52-1b88-45aa-9ab1-c0a712bf733e-kube-api-access-cqrv7\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:16:19.200560 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.200541    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h52sl\" (UniqueName: \"kubernetes.io/projected/77e0a9a6-4d92-4f21-8ecf-7b791b33a730-kube-api-access-h52sl\") pod \"thanos-querier-64cb9dd857-hf26d\" (UID: \"77e0a9a6-4d92-4f21-8ecf-7b791b33a730\") " pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:19.200677 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.200624    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4fxt\" (UniqueName: \"kubernetes.io/projected/13b111e6-49a0-45e4-bb33-a49c5dc4d439-kube-api-access-n4fxt\") pod \"kube-state-metrics-5dc8d7d89b-6qqhd\" (UID: \"13b111e6-49a0-45e4-bb33-a49c5dc4d439\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6qqhd"
Nov 04 07:16:19.200804 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.200784    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7hnk\" (UniqueName: \"kubernetes.io/projected/fe2e0b0d-3d26-4d4d-90a6-ef499452293b-kube-api-access-q7hnk\") pod \"openshift-state-metrics-7bd7489957-k6tsc\" (UID: \"fe2e0b0d-3d26-4d4d-90a6-ef499452293b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-k6tsc"
Nov 04 07:16:19.200880 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.200863    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rt4f\" (UniqueName: \"kubernetes.io/projected/542b9c39-a718-4064-9640-5123b2dcb358-kube-api-access-7rt4f\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.205100 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.205080    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd74j\" (UniqueName: \"kubernetes.io/projected/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-kube-api-access-xd74j\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:16:19.205193 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.205143    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zrpx\" (UniqueName: \"kubernetes.io/projected/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-kube-api-access-5zrpx\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:16:19.205400 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.205374    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxgcd\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-kube-api-access-mxgcd\") pod \"prometheus-k8s-0\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:19.276553 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276522    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-host\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.276683 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276555    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysconfig\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.276683 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276580    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-etc-kubernetes\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.276683 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276618    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-etc-kubernetes\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.276683 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276643    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-system-cni-dir\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276687    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-netns\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276651    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-host\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276690    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-system-cni-dir\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276651    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysconfig\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276718    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-dbus\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276723    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-netns\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276738    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-k8s-cni-cncf-io\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276762    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-k8s-cni-cncf-io\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276767    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-sys\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276772    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-dbus\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276794    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-sys\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276796    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-system-cni-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276826    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-system-cni-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.276833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276832    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276853    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysctl-conf\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276878    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-socket-dir-parent\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276889    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276901    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-slash\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276926    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-systemd\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276931    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-slash\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276950    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276967    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-systemd\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276977    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-cni-bin\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276980    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysctl-conf\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.276984    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-socket-dir-parent\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277011    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/723c3b52-1b88-45aa-9ab1-c0a712bf733e-crio-socket\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277018    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-cni-bin\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277035    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277051    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-host\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277073    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-modprobe-d\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277086    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-kubernetes\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277092    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f1d3530-4c03-4633-92b1-9fc2ab954b9f-host\") pod \"node-ca-2pxh9\" (UID: \"3f1d3530-4c03-4633-92b1-9fc2ab954b9f\") " pod="openshift-image-registry/node-ca-2pxh9"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277100    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-systemd-units\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277122    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-wtmp\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277143    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-systemd-units\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277148    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-kubernetes\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277053    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/723c3b52-1b88-45aa-9ab1-c0a712bf733e-crio-socket\") pod \"insights-runtime-extractor-5zbxq\" (UID: \"723c3b52-1b88-45aa-9ab1-c0a712bf733e\") " pod="openshift-insights/insights-runtime-extractor-5zbxq"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277147    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-var-lib-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277178    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-modprobe-d\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277188    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-log-socket\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277206    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-cni-netd\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277210    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-node-exporter-wtmp\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277178    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-var-lib-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277237    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-cni-netd\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277236    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-cnibin\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.277276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277272    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-log-socket\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277307    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-cnibin\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277310    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-kubelet\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277326    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-kubelet\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277334    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-root\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277348    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysctl-d\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277381    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-systemd\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277403    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-sys\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277424    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-lib-modules\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277435    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-systemd\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277402    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fff3c48c-f8c5-4946-a95e-b5faedb4dab8-root\") pod \"node-exporter-bwnvs\" (UID: \"fff3c48c-f8c5-4946-a95e-b5faedb4dab8\") " pod="openshift-monitoring/node-exporter-bwnvs"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277447    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-node-log\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277458    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-etc-sysctl-d\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277442    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-sys\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277476    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-node-log\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277470    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cnibin\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277500    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-cnibin\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277510    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-sys-fs\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277531    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-lib-modules\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277539    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-host-slash\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277563    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277567    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6-host-slash\") pod \"iptables-alerter-qt2pw\" (UID: \"1f9ab2f4-8f74-484b-b0ed-ac13ff8e26c6\") " pod="openshift-network-operator/iptables-alerter-qt2pw"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277567    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-sys-fs\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277586    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-var-lib-kubelet\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277600    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d74bcca-3fbc-4984-8acf-b53b0f34e30a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k7449\" (UID: \"0d74bcca-3fbc-4984-8acf-b53b0f34e30a\") " pod="openshift-multus/multus-additional-cni-plugins-k7449"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277605    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-device-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277627    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-conf-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277633    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-device-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277656    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-conf-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277657    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277683    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-cni-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277687    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-var-lib-kubelet\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277697    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-etc-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277714    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-socket-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277728    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-multus-cni-dir\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277688    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277737    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-ovn\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277744    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-etc-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277760    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277772    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-socket-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277781    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-ovn\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277782    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-cni-bin\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277796    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277809    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-cni-bin\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277811    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-registration-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277844    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-kubelet-config\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277849    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/07aa084c-c07a-4901-800d-a3ba2a659bd5-registration-dir\") pod \"aws-ebs-csi-driver-node-7w26m\" (UID: \"07aa084c-c07a-4901-800d-a3ba2a659bd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w26m"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277868    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-hostroot\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277891    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bd7ea631-0ca9-4829-8408-8352e1d1bc7f-kubelet-config\") pod \"global-pull-secret-syncer-dxhns\" (UID: \"bd7ea631-0ca9-4829-8408-8352e1d1bc7f\") " pod="kube-system/global-pull-secret-syncer-dxhns"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277895    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277921    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-run\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277925    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-hostroot\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277942    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-cni-multus\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277953    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-run-openvswitch\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277965    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03300026-6052-474a-a6b0-53afd4f3f67a-run\") pod \"tuned-758f6\" (UID: \"03300026-6052-474a-a6b0-53afd4f3f67a\") " pod="openshift-cluster-node-tuning-operator/tuned-758f6"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277971    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f95c6e6a-6787-423e-8129-a62feb0a5156-hosts-file\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277978    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-var-lib-cni-multus\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.277998    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-kubelet\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.278011    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f95c6e6a-6787-423e-8129-a62feb0a5156-hosts-file\") pod \"node-resolver-2zd55\" (UID: \"f95c6e6a-6787-423e-8129-a62feb0a5156\") " pod="openshift-dns/node-resolver-2zd55"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.278025    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-run-netns\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.278040    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-multus-certs\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.278076    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-kubelet\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.278091    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/542b9c39-a718-4064-9640-5123b2dcb358-host-run-netns\") pod \"ovnkube-node-2gcvm\" (UID: \"542b9c39-a718-4064-9640-5123b2dcb358\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:19.278237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.278115    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/caee6966-5544-44a7-8b90-c431ef6b94de-host-run-multus-certs\") pod \"multus-dn8mz\" (UID: \"caee6966-5544-44a7-8b90-c431ef6b94de\") " pod="openshift-multus/multus-dn8mz"
Nov 04 07:16:19.920583 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:19.920547    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2w47v"
Nov 04 07:16:21.021968 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:21.021934    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:22.706763 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:22.706715    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:16:23.545639 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:23.545605    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-64cb9dd857-hf26d"
Nov 04 07:16:23.721265 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:23.721235    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:23.721659 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:23.721449    5626 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 07:16:24.048172 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:24.048148    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-2xqz7"
Nov 04 07:16:24.187412 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:24.187381    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:28.244070 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:28.244028    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-76495f7c59-rsjg4"
Nov 04 07:16:36.171409 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:36.171303    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:16:37.843246 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:37.843209    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 07:16:37.844499 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:37.844444    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="prometheus" containerID="cri-o://085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1" gracePeriod=600
Nov 04 07:16:37.845380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:37.844886    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="thanos-sidecar" containerID="cri-o://ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381" gracePeriod=600
Nov 04 07:16:37.845380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:37.844881    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="kube-rbac-proxy-web" containerID="cri-o://359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b" gracePeriod=600
Nov 04 07:16:37.845380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:37.845007    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="config-reloader" containerID="cri-o://e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd" gracePeriod=600
Nov 04 07:16:37.845380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:37.845096    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918" gracePeriod=600
Nov 04 07:16:37.845380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:37.845149    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="kube-rbac-proxy" containerID="cri-o://e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403" gracePeriod=600
Nov 04 07:16:38.036550 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.036523    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-knhlk"
Nov 04 07:16:38.080869 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.080845    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.224082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.223989    5626 generic.go:356] "Generic (PLEG): container finished" podID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerID="e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918" exitCode=0
Nov 04 07:16:38.224082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224021    5626 generic.go:356] "Generic (PLEG): container finished" podID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerID="e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403" exitCode=0
Nov 04 07:16:38.224082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224031    5626 generic.go:356] "Generic (PLEG): container finished" podID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerID="359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b" exitCode=0
Nov 04 07:16:38.224082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224040    5626 generic.go:356] "Generic (PLEG): container finished" podID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerID="ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381" exitCode=0
Nov 04 07:16:38.224082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224047    5626 generic.go:356] "Generic (PLEG): container finished" podID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerID="e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd" exitCode=0
Nov 04 07:16:38.224082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224056    5626 generic.go:356] "Generic (PLEG): container finished" podID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerID="085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1" exitCode=0
Nov 04 07:16:38.224082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224063    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerDied","Data":"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"}
Nov 04 07:16:38.224455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224089    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.224455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224127    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerDied","Data":"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"}
Nov 04 07:16:38.224455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224140    5626 scope.go:122] "RemoveContainer" containerID="e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"
Nov 04 07:16:38.224455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224155    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerDied","Data":"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"}
Nov 04 07:16:38.224455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224180    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerDied","Data":"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"}
Nov 04 07:16:38.224455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224202    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerDied","Data":"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"}
Nov 04 07:16:38.224455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224228    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerDied","Data":"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"}
Nov 04 07:16:38.224455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.224255    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb89bad1-c644-4d86-b4bf-0827628ac4d3","Type":"ContainerDied","Data":"6d63d36dfd91395a2a336c91682cff45aea0eba596c6c8e305a4a618d84bdb46"}
Nov 04 07:16:38.226296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226276    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-thanos-prometheus-http-client-file\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226311    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-tls\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226331    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226348    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226397    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-1\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226423    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-tls-assets\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-tls-assets\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226439    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226498    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-db\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226526    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-0\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226548    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config-out\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226567    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-kube-rbac-proxy\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226599    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-2\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226617    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-kubelet-serving-ca-bundle\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226636    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-kube-api-access-mxgcd\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-kube-api-access-mxgcd\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226658    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-web-config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-web-config\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.226756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226693    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-metrics-client-ca\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.227191 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.226937    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-1" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:16:38.227191 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.227023    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-kubelet-serving-ca-bundle" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:16:38.227600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.227354    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-2" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:16:38.227849 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.227829    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-serving-certs-ca-bundle\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.228029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.227999    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-trusted-ca-bundle\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.228152 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.228051    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-grpc-tls\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.228152 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.228089    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-metrics-client-certs\") pod \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\" (UID: \"fb89bad1-c644-4d86-b4bf-0827628ac4d3\") "
Nov 04 07:16:38.228399 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.228380    5626 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.228797 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.228598    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-db" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:16:38.229565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.228941    5626 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.229565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.228978    5626 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.229565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.229240    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-0" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:16:38.230616 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.230592    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-thanos-prometheus-http-client-file" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:16:38.230751 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.230677    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-kube-rbac-proxy-web" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:16:38.231310 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.231286    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-serving-certs-ca-bundle" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:16:38.231522 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.231503    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-trusted-ca-bundle" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:16:38.231654 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.231633    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-metrics-client-ca" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:16:38.233415 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.233217    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-tls" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:16:38.233415 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.233375    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-tls-assets" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:16:38.233559 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.233518    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-kube-rbac-proxy" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:16:38.233756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.233728    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-metrics-client-certs" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:16:38.233831 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.233772    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-grpc-tls" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:16:38.234682 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.234653    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config-out" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:16:38.234768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.234680    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-thanos-sidecar-tls" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:16:38.234768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.234687    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-kube-api-access-mxgcd" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "kube-api-access-mxgcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:16:38.234869 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.234769    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:16:38.238636 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.238617    5626 scope.go:122] "RemoveContainer" containerID="e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"
Nov 04 07:16:38.245677 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.245657    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-web-config" pod "fb89bad1-c644-4d86-b4bf-0827628ac4d3" (UID: "fb89bad1-c644-4d86-b4bf-0827628ac4d3"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:16:38.245827 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.245813    5626 scope.go:122] "RemoveContainer" containerID="359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"
Nov 04 07:16:38.253212 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.253197    5626 scope.go:122] "RemoveContainer" containerID="ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"
Nov 04 07:16:38.259769 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.259755    5626 scope.go:122] "RemoveContainer" containerID="e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"
Nov 04 07:16:38.266069 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.266049    5626 scope.go:122] "RemoveContainer" containerID="085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"
Nov 04 07:16:38.274283 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.274265    5626 scope.go:122] "RemoveContainer" containerID="37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"
Nov 04 07:16:38.281659 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.281641    5626 scope.go:122] "RemoveContainer" containerID="e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"
Nov 04 07:16:38.281938 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:38.281917    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": container with ID starting with e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918 not found: ID does not exist" containerID="e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"
Nov 04 07:16:38.282010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.281952    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"} err="failed to get container status \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": rpc error: code = NotFound desc = could not find container \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": container with ID starting with e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918 not found: ID does not exist"
Nov 04 07:16:38.282010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.281994    5626 scope.go:122] "RemoveContainer" containerID="e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"
Nov 04 07:16:38.282255 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:38.282238    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": container with ID starting with e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403 not found: ID does not exist" containerID="e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"
Nov 04 07:16:38.282309 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.282260    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"} err="failed to get container status \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": rpc error: code = NotFound desc = could not find container \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": container with ID starting with e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403 not found: ID does not exist"
Nov 04 07:16:38.282309 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.282274    5626 scope.go:122] "RemoveContainer" containerID="359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"
Nov 04 07:16:38.282513 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:38.282495    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": container with ID starting with 359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b not found: ID does not exist" containerID="359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"
Nov 04 07:16:38.282561 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.282519    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"} err="failed to get container status \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": rpc error: code = NotFound desc = could not find container \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": container with ID starting with 359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b not found: ID does not exist"
Nov 04 07:16:38.282561 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.282533    5626 scope.go:122] "RemoveContainer" containerID="ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"
Nov 04 07:16:38.282771 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:38.282755    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": container with ID starting with ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381 not found: ID does not exist" containerID="ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"
Nov 04 07:16:38.282813 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.282775    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"} err="failed to get container status \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": rpc error: code = NotFound desc = could not find container \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": container with ID starting with ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381 not found: ID does not exist"
Nov 04 07:16:38.282813 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.282788    5626 scope.go:122] "RemoveContainer" containerID="e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"
Nov 04 07:16:38.282989 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:38.282972    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": container with ID starting with e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd not found: ID does not exist" containerID="e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"
Nov 04 07:16:38.283033 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.282993    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"} err="failed to get container status \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": rpc error: code = NotFound desc = could not find container \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": container with ID starting with e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd not found: ID does not exist"
Nov 04 07:16:38.283033 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.283007    5626 scope.go:122] "RemoveContainer" containerID="085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"
Nov 04 07:16:38.283204 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:38.283186    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": container with ID starting with 085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1 not found: ID does not exist" containerID="085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"
Nov 04 07:16:38.283247 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.283206    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"} err="failed to get container status \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": rpc error: code = NotFound desc = could not find container \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": container with ID starting with 085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1 not found: ID does not exist"
Nov 04 07:16:38.283247 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.283217    5626 scope.go:122] "RemoveContainer" containerID="37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"
Nov 04 07:16:38.283489 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:16:38.283473    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": container with ID starting with 37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7 not found: ID does not exist" containerID="37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"
Nov 04 07:16:38.283592 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.283497    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"} err="failed to get container status \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": rpc error: code = NotFound desc = could not find container \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": container with ID starting with 37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7 not found: ID does not exist"
Nov 04 07:16:38.283592 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.283515    5626 scope.go:122] "RemoveContainer" containerID="e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"
Nov 04 07:16:38.283736 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.283720    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"} err="failed to get container status \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": rpc error: code = NotFound desc = could not find container \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": container with ID starting with e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918 not found: ID does not exist"
Nov 04 07:16:38.283779 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.283735    5626 scope.go:122] "RemoveContainer" containerID="e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"
Nov 04 07:16:38.283979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.283962    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"} err="failed to get container status \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": rpc error: code = NotFound desc = could not find container \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": container with ID starting with e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403 not found: ID does not exist"
Nov 04 07:16:38.284037 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.283979    5626 scope.go:122] "RemoveContainer" containerID="359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"
Nov 04 07:16:38.284178 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.284160    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"} err="failed to get container status \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": rpc error: code = NotFound desc = could not find container \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": container with ID starting with 359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b not found: ID does not exist"
Nov 04 07:16:38.284231 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.284177    5626 scope.go:122] "RemoveContainer" containerID="ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"
Nov 04 07:16:38.284410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.284392    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"} err="failed to get container status \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": rpc error: code = NotFound desc = could not find container \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": container with ID starting with ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381 not found: ID does not exist"
Nov 04 07:16:38.284468 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.284410    5626 scope.go:122] "RemoveContainer" containerID="e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"
Nov 04 07:16:38.284593 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.284579    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"} err="failed to get container status \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": rpc error: code = NotFound desc = could not find container \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": container with ID starting with e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd not found: ID does not exist"
Nov 04 07:16:38.284593 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.284592    5626 scope.go:122] "RemoveContainer" containerID="085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"
Nov 04 07:16:38.284797 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.284777    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"} err="failed to get container status \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": rpc error: code = NotFound desc = could not find container \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": container with ID starting with 085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1 not found: ID does not exist"
Nov 04 07:16:38.284839 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.284798    5626 scope.go:122] "RemoveContainer" containerID="37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"
Nov 04 07:16:38.284987 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.284968    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"} err="failed to get container status \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": rpc error: code = NotFound desc = could not find container \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": container with ID starting with 37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7 not found: ID does not exist"
Nov 04 07:16:38.285034 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.284987    5626 scope.go:122] "RemoveContainer" containerID="e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"
Nov 04 07:16:38.285224 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.285205    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"} err="failed to get container status \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": rpc error: code = NotFound desc = could not find container \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": container with ID starting with e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918 not found: ID does not exist"
Nov 04 07:16:38.285265 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.285226    5626 scope.go:122] "RemoveContainer" containerID="e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"
Nov 04 07:16:38.285471 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.285451    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"} err="failed to get container status \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": rpc error: code = NotFound desc = could not find container \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": container with ID starting with e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403 not found: ID does not exist"
Nov 04 07:16:38.285471 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.285469    5626 scope.go:122] "RemoveContainer" containerID="359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"
Nov 04 07:16:38.285661 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.285646    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"} err="failed to get container status \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": rpc error: code = NotFound desc = could not find container \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": container with ID starting with 359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b not found: ID does not exist"
Nov 04 07:16:38.285700 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.285662    5626 scope.go:122] "RemoveContainer" containerID="ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"
Nov 04 07:16:38.285844 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.285827    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"} err="failed to get container status \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": rpc error: code = NotFound desc = could not find container \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": container with ID starting with ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381 not found: ID does not exist"
Nov 04 07:16:38.285888 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.285845    5626 scope.go:122] "RemoveContainer" containerID="e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"
Nov 04 07:16:38.286106 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.286089    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"} err="failed to get container status \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": rpc error: code = NotFound desc = could not find container \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": container with ID starting with e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd not found: ID does not exist"
Nov 04 07:16:38.286174 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.286108    5626 scope.go:122] "RemoveContainer" containerID="085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"
Nov 04 07:16:38.286321 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.286303    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"} err="failed to get container status \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": rpc error: code = NotFound desc = could not find container \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": container with ID starting with 085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1 not found: ID does not exist"
Nov 04 07:16:38.286386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.286322    5626 scope.go:122] "RemoveContainer" containerID="37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"
Nov 04 07:16:38.286547 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.286530    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"} err="failed to get container status \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": rpc error: code = NotFound desc = could not find container \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": container with ID starting with 37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7 not found: ID does not exist"
Nov 04 07:16:38.286590 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.286549    5626 scope.go:122] "RemoveContainer" containerID="e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"
Nov 04 07:16:38.286739 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.286722    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"} err="failed to get container status \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": rpc error: code = NotFound desc = could not find container \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": container with ID starting with e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918 not found: ID does not exist"
Nov 04 07:16:38.286783 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.286739    5626 scope.go:122] "RemoveContainer" containerID="e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"
Nov 04 07:16:38.286941 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.286922    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"} err="failed to get container status \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": rpc error: code = NotFound desc = could not find container \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": container with ID starting with e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403 not found: ID does not exist"
Nov 04 07:16:38.287008 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.286941    5626 scope.go:122] "RemoveContainer" containerID="359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"
Nov 04 07:16:38.287199 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.287182    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"} err="failed to get container status \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": rpc error: code = NotFound desc = could not find container \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": container with ID starting with 359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b not found: ID does not exist"
Nov 04 07:16:38.287240 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.287199    5626 scope.go:122] "RemoveContainer" containerID="ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"
Nov 04 07:16:38.287398 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.287381    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"} err="failed to get container status \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": rpc error: code = NotFound desc = could not find container \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": container with ID starting with ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381 not found: ID does not exist"
Nov 04 07:16:38.287468 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.287399    5626 scope.go:122] "RemoveContainer" containerID="e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"
Nov 04 07:16:38.287632 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.287615    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"} err="failed to get container status \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": rpc error: code = NotFound desc = could not find container \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": container with ID starting with e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd not found: ID does not exist"
Nov 04 07:16:38.287670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.287632    5626 scope.go:122] "RemoveContainer" containerID="085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"
Nov 04 07:16:38.287888 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.287871    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"} err="failed to get container status \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": rpc error: code = NotFound desc = could not find container \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": container with ID starting with 085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1 not found: ID does not exist"
Nov 04 07:16:38.287926 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.287889    5626 scope.go:122] "RemoveContainer" containerID="37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"
Nov 04 07:16:38.288105 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.288089    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"} err="failed to get container status \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": rpc error: code = NotFound desc = could not find container \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": container with ID starting with 37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7 not found: ID does not exist"
Nov 04 07:16:38.288150 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.288104    5626 scope.go:122] "RemoveContainer" containerID="e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"
Nov 04 07:16:38.288335 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.288316    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"} err="failed to get container status \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": rpc error: code = NotFound desc = could not find container \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": container with ID starting with e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918 not found: ID does not exist"
Nov 04 07:16:38.288335 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.288333    5626 scope.go:122] "RemoveContainer" containerID="e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"
Nov 04 07:16:38.288578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.288561    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"} err="failed to get container status \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": rpc error: code = NotFound desc = could not find container \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": container with ID starting with e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403 not found: ID does not exist"
Nov 04 07:16:38.288578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.288578    5626 scope.go:122] "RemoveContainer" containerID="359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"
Nov 04 07:16:38.288776 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.288759    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"} err="failed to get container status \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": rpc error: code = NotFound desc = could not find container \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": container with ID starting with 359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b not found: ID does not exist"
Nov 04 07:16:38.288819 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.288776    5626 scope.go:122] "RemoveContainer" containerID="ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"
Nov 04 07:16:38.288972 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.288953    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"} err="failed to get container status \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": rpc error: code = NotFound desc = could not find container \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": container with ID starting with ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381 not found: ID does not exist"
Nov 04 07:16:38.289017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.288972    5626 scope.go:122] "RemoveContainer" containerID="e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"
Nov 04 07:16:38.289161 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.289146    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"} err="failed to get container status \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": rpc error: code = NotFound desc = could not find container \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": container with ID starting with e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd not found: ID does not exist"
Nov 04 07:16:38.289197 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.289160    5626 scope.go:122] "RemoveContainer" containerID="085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"
Nov 04 07:16:38.289375 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.289345    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"} err="failed to get container status \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": rpc error: code = NotFound desc = could not find container \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": container with ID starting with 085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1 not found: ID does not exist"
Nov 04 07:16:38.289448 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.289386    5626 scope.go:122] "RemoveContainer" containerID="37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"
Nov 04 07:16:38.289596 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.289578    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"} err="failed to get container status \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": rpc error: code = NotFound desc = could not find container \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": container with ID starting with 37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7 not found: ID does not exist"
Nov 04 07:16:38.289596 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.289594    5626 scope.go:122] "RemoveContainer" containerID="e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"
Nov 04 07:16:38.289794 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.289779    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918"} err="failed to get container status \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": rpc error: code = NotFound desc = could not find container \"e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918\": container with ID starting with e6bb2fd38b5143ae60caa9cfbd67e593412704194c420ad06e72a654a76b6918 not found: ID does not exist"
Nov 04 07:16:38.289857 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.289794    5626 scope.go:122] "RemoveContainer" containerID="e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"
Nov 04 07:16:38.289994 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.289977    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403"} err="failed to get container status \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": rpc error: code = NotFound desc = could not find container \"e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403\": container with ID starting with e8b6e272c30e85d91f1bd320b3629dc475e491c221d6464ad1ac249cdc07d403 not found: ID does not exist"
Nov 04 07:16:38.290041 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.289996    5626 scope.go:122] "RemoveContainer" containerID="359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"
Nov 04 07:16:38.290182 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.290168    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b"} err="failed to get container status \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": rpc error: code = NotFound desc = could not find container \"359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b\": container with ID starting with 359c08219fd5766e82e577ef77c636464e2df5c5462512512b7a9a170c09b87b not found: ID does not exist"
Nov 04 07:16:38.290182 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.290181    5626 scope.go:122] "RemoveContainer" containerID="ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"
Nov 04 07:16:38.290417 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.290401    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381"} err="failed to get container status \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": rpc error: code = NotFound desc = could not find container \"ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381\": container with ID starting with ef334f95227c3f7b5bfd97a5694e5f3e140bab01418d6fd8d3a71006f4a71381 not found: ID does not exist"
Nov 04 07:16:38.290490 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.290418    5626 scope.go:122] "RemoveContainer" containerID="e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"
Nov 04 07:16:38.290655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.290638    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd"} err="failed to get container status \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": rpc error: code = NotFound desc = could not find container \"e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd\": container with ID starting with e90817a9384a1a8b74339ca9e3bcc0a6d4c14b261f8e2efca2046e0b9420dadd not found: ID does not exist"
Nov 04 07:16:38.290702 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.290656    5626 scope.go:122] "RemoveContainer" containerID="085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"
Nov 04 07:16:38.290848 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.290832    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1"} err="failed to get container status \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": rpc error: code = NotFound desc = could not find container \"085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1\": container with ID starting with 085316f06a67f0872356d55dad1b8528da22bad9756734095e7c527f34731df1 not found: ID does not exist"
Nov 04 07:16:38.290915 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.290849    5626 scope.go:122] "RemoveContainer" containerID="37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"
Nov 04 07:16:38.291062 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.291044    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7"} err="failed to get container status \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": rpc error: code = NotFound desc = could not find container \"37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7\": container with ID starting with 37d610057056d9f1f6de11ee37d455d253d9350ffcd45e46ede00d61fd6324b7 not found: ID does not exist"
Nov 04 07:16:38.329439 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329418    5626 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-metrics-client-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329439 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329438    5626 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329447    5626 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329457    5626 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-grpc-tls\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329466    5626 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-metrics-client-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329474    5626 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329484    5626 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329493    5626 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329503    5626 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329511    5626 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-tls-assets\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329520    5626 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329528    5626 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-db\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329536    5626 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fb89bad1-c644-4d86-b4bf-0827628ac4d3-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329828 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329544    5626 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb89bad1-c644-4d86-b4bf-0827628ac4d3-config-out\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329828 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329552    5626 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-secret-kube-rbac-proxy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329828 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329560    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxgcd\" (UniqueName: \"kubernetes.io/projected/fb89bad1-c644-4d86-b4bf-0827628ac4d3-kube-api-access-mxgcd\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.329828 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.329568    5626 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb89bad1-c644-4d86-b4bf-0827628ac4d3-web-config\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:16:38.551681 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.551655    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 07:16:38.571456 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.569993    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 07:16:38.604646 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604620    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 07:16:38.604904 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604891    5626 cpu_manager.go:414] "containerMap: removing container" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="kube-rbac-proxy"
Nov 04 07:16:38.604957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604904    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="kube-rbac-proxy"
Nov 04 07:16:38.604957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604919    5626 cpu_manager.go:414] "containerMap: removing container" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="thanos-sidecar"
Nov 04 07:16:38.604957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604923    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="thanos-sidecar"
Nov 04 07:16:38.604957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604930    5626 cpu_manager.go:414] "containerMap: removing container" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="init-config-reloader"
Nov 04 07:16:38.604957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604934    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="init-config-reloader"
Nov 04 07:16:38.604957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604943    5626 cpu_manager.go:414] "containerMap: removing container" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="kube-rbac-proxy-web"
Nov 04 07:16:38.604957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604948    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="kube-rbac-proxy-web"
Nov 04 07:16:38.604957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604959    5626 cpu_manager.go:414] "containerMap: removing container" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="kube-rbac-proxy-thanos"
Nov 04 07:16:38.605206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604964    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="kube-rbac-proxy-thanos"
Nov 04 07:16:38.605206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604974    5626 cpu_manager.go:414] "containerMap: removing container" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="config-reloader"
Nov 04 07:16:38.605206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604978    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="config-reloader"
Nov 04 07:16:38.605206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604985    5626 cpu_manager.go:414] "containerMap: removing container" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="prometheus"
Nov 04 07:16:38.605206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.604989    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="prometheus"
Nov 04 07:16:38.605206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.605030    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="kube-rbac-proxy-thanos"
Nov 04 07:16:38.605206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.605037    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="config-reloader"
Nov 04 07:16:38.605206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.605043    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="prometheus"
Nov 04 07:16:38.605206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.605050    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="kube-rbac-proxy"
Nov 04 07:16:38.605206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.605056    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="thanos-sidecar"
Nov 04 07:16:38.605206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.605062    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="init-config-reloader"
Nov 04 07:16:38.605206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.605067    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" containerName="kube-rbac-proxy-web"
Nov 04 07:16:38.608511 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.608494    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.617204 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.617181    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 04 07:16:38.617322 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.617239    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 04 07:16:38.617447 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.617410    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 04 07:16:38.619378 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.619341    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 04 07:16:38.620444 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.620429    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 04 07:16:38.622618 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.622602    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 04 07:16:38.622722 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.622673    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vb7ng\""
Nov 04 07:16:38.622722 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.622693    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 04 07:16:38.622722 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.622717    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 04 07:16:38.622910 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.622758    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7a9sgmd41l2bu\""
Nov 04 07:16:38.622910 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.622766    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 04 07:16:38.622910 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.622771    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 04 07:16:38.622910 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.622767    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 04 07:16:38.622910 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.622788    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 04 07:16:38.622910 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.622809    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 04 07:16:38.626009 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.625992    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 04 07:16:38.637845 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.637823    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 07:16:38.731890 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.731841    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.731890 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.731892    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732116 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.731924    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-config\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732116 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.731948    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732116 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.731998    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732116 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732095    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732313 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732156    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732313 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732194    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732313 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732225    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7cgg\" (UniqueName: \"kubernetes.io/projected/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-kube-api-access-f7cgg\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732313 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732258    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732313 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732296    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732523 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732322    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732523 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732407    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732523 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732442    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732523 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732464    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732523 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732504    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732535    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732603    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732640    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.732764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.732670    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833301 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833220    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833301 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833267    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833301 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833292    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833321    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833343    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833394    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-config\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833417    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833439    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833477    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833506    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833540    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833569    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7cgg\" (UniqueName: \"kubernetes.io/projected/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-kube-api-access-f7cgg\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833603    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833630    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833669    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833679    5626 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 07:16:38.833929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833701    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833741    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833771    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833799    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.833929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.833827    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.834397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.834051    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.836106 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.834512    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.836106 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.835074    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.836106 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.835338    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.836106 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.836016    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.836407 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.836255    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.836407 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.836336    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.836614 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.836590    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.837282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.836839    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-config\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.837282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.836999    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.837282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.837069    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.837282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.837072    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.837515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.837408    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.838314 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.838298    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.838673 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.838650    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.838846 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.838827    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.839276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.839256    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.839319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.839280    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.839904 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.839887    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.847002 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.846976    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7cgg\" (UniqueName: \"kubernetes.io/projected/4a1cfe38-f37a-49e8-a29a-16c6bdb44759-kube-api-access-f7cgg\") pod \"prometheus-k8s-0\" (UID: \"4a1cfe38-f37a-49e8-a29a-16c6bdb44759\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:38.924026 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:38.924006    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:39.209919 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:39.209683    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 07:16:39.212521 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:16:39.212485    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a1cfe38_f37a_49e8_a29a_16c6bdb44759.slice/crio-e544e1efeacad305f24097753453f70009bc05b29f6d26db207a2782a3c9aec7 WatchSource:0}: Error finding container e544e1efeacad305f24097753453f70009bc05b29f6d26db207a2782a3c9aec7: Status 404 returned error can't find the container with id e544e1efeacad305f24097753453f70009bc05b29f6d26db207a2782a3c9aec7
Nov 04 07:16:39.230349 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:39.230324    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a1cfe38-f37a-49e8-a29a-16c6bdb44759","Type":"ContainerStarted","Data":"e544e1efeacad305f24097753453f70009bc05b29f6d26db207a2782a3c9aec7"}
Nov 04 07:16:40.090279 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:40.090234    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb89bad1-c644-4d86-b4bf-0827628ac4d3" path="/var/lib/kubelet/pods/fb89bad1-c644-4d86-b4bf-0827628ac4d3/volumes"
Nov 04 07:16:40.235167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:40.235132    5626 generic.go:356] "Generic (PLEG): container finished" podID="4a1cfe38-f37a-49e8-a29a-16c6bdb44759" containerID="81305b5d2eae617157801198dfb8ebdd5def21aa6fe1e4a736703a7e8539c608" exitCode=0
Nov 04 07:16:40.235347 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:40.235203    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a1cfe38-f37a-49e8-a29a-16c6bdb44759","Type":"ContainerDied","Data":"81305b5d2eae617157801198dfb8ebdd5def21aa6fe1e4a736703a7e8539c608"}
Nov 04 07:16:41.242507 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:41.242471    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a1cfe38-f37a-49e8-a29a-16c6bdb44759","Type":"ContainerStarted","Data":"a4884edf4c1810f32fafdc204533ff2a051e5f0d36734a20efb504551c266adf"}
Nov 04 07:16:41.242964 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:41.242559    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a1cfe38-f37a-49e8-a29a-16c6bdb44759","Type":"ContainerStarted","Data":"570310235ec6358028a27adec10a2bd63f1476f319837036c895da73501abd60"}
Nov 04 07:16:41.242964 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:41.242587    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a1cfe38-f37a-49e8-a29a-16c6bdb44759","Type":"ContainerStarted","Data":"ad8fcdfd5fe1dd9ea70f017f9c667f4851dfd8ede34d1534f4ae207d9e4faec8"}
Nov 04 07:16:41.242964 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:41.242614    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a1cfe38-f37a-49e8-a29a-16c6bdb44759","Type":"ContainerStarted","Data":"37505b0de585dc58e5f470bffc30cd58ae954d89b5039105158968be0da3a541"}
Nov 04 07:16:41.242964 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:41.242642    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a1cfe38-f37a-49e8-a29a-16c6bdb44759","Type":"ContainerStarted","Data":"43b5d70920487890d7dc62b8e4a9690107c11e11085aeac108d81daa3a2cfd88"}
Nov 04 07:16:41.242964 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:41.242664    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a1cfe38-f37a-49e8-a29a-16c6bdb44759","Type":"ContainerStarted","Data":"117ecf6ae1be873385a3fd0534e7a8570fde56913208fe7abee19c7c43af2b41"}
Nov 04 07:16:41.287105 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:41.287049    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.270836022 podStartE2EDuration="3.287034413s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.198391ms" isStatefulPod=false podCreationTimestamp="2026-11-04 07:16:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:16:41.281560907 +0000 UTC m=+23.303776177" watchObservedRunningTime="2026-11-04 07:16:41.287034413 +0000 UTC m=+23.309249685"
Nov 04 07:16:43.924829 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:43.924800    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:49.252301 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:49.252271    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gcvm"
Nov 04 07:16:52.709234 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:52.709203    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-smfct"
Nov 04 07:16:57.458445 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:57.458405    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:57.473337 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:57.473316    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:16:58.320928 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:16:58.320899    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 07:17:06.170652 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:17:06.170610    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:17:36.170735 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:17:36.170696    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:18:06.171665 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:18:06.171579    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:18:36.172077 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:18:36.172040    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:19:06.171492 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:19:06.171456    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:19:25.464117 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.464080    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"]
Nov 04 07:19:25.467515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.467493    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:25.471328 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.471310    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 04 07:19:25.472630 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.472605    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 04 07:19:25.472761 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.472668    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-hsk9c\""
Nov 04 07:19:25.477276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.477255    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"]
Nov 04 07:19:25.548108 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.548070    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j99r4\" (UniqueName: \"kubernetes.io/projected/8b2301ed-5209-45f7-8262-51fb2f80ac33-kube-api-access-j99r4\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk\" (UID: \"8b2301ed-5209-45f7-8262-51fb2f80ac33\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:25.548342 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.548113    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk\" (UID: \"8b2301ed-5209-45f7-8262-51fb2f80ac33\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:25.548342 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.548201    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk\" (UID: \"8b2301ed-5209-45f7-8262-51fb2f80ac33\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:25.649330 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.649299    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j99r4\" (UniqueName: \"kubernetes.io/projected/8b2301ed-5209-45f7-8262-51fb2f80ac33-kube-api-access-j99r4\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk\" (UID: \"8b2301ed-5209-45f7-8262-51fb2f80ac33\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:25.649450 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.649336    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk\" (UID: \"8b2301ed-5209-45f7-8262-51fb2f80ac33\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:25.649450 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.649378    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk\" (UID: \"8b2301ed-5209-45f7-8262-51fb2f80ac33\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:25.649715 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.649697    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk\" (UID: \"8b2301ed-5209-45f7-8262-51fb2f80ac33\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:25.649787 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.649738    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk\" (UID: \"8b2301ed-5209-45f7-8262-51fb2f80ac33\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:25.662148 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.662120    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j99r4\" (UniqueName: \"kubernetes.io/projected/8b2301ed-5209-45f7-8262-51fb2f80ac33-kube-api-access-j99r4\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk\" (UID: \"8b2301ed-5209-45f7-8262-51fb2f80ac33\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:25.784769 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:25.784741    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:26.056420 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:26.056337    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"]
Nov 04 07:19:26.059942 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:19:26.059916    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b2301ed_5209_45f7_8262_51fb2f80ac33.slice/crio-50af61bdd20ae7bb1155c454322acbb55f3c219655ed0b70517975be8963a0d4 WatchSource:0}: Error finding container 50af61bdd20ae7bb1155c454322acbb55f3c219655ed0b70517975be8963a0d4: Status 404 returned error can't find the container with id 50af61bdd20ae7bb1155c454322acbb55f3c219655ed0b70517975be8963a0d4
Nov 04 07:19:26.061936 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:26.061920    5626 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 07:19:26.820344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:26.820309    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk" event={"ID":"8b2301ed-5209-45f7-8262-51fb2f80ac33","Type":"ContainerStarted","Data":"50af61bdd20ae7bb1155c454322acbb55f3c219655ed0b70517975be8963a0d4"}
Nov 04 07:19:27.386678 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.386644    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"]
Nov 04 07:19:27.390056 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.390033    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:27.395202 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.395165    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-p29m7\""
Nov 04 07:19:27.400733 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.400712    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"]
Nov 04 07:19:27.463122 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.463096    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6x5d\" (UniqueName: \"kubernetes.io/projected/9c102465-8050-4a38-8d87-a8149acf7f9f-kube-api-access-r6x5d\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9\" (UID: \"9c102465-8050-4a38-8d87-a8149acf7f9f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:27.463287 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.463158    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9\" (UID: \"9c102465-8050-4a38-8d87-a8149acf7f9f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:27.463383 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.463352    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9\" (UID: \"9c102465-8050-4a38-8d87-a8149acf7f9f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:27.486833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.486804    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"]
Nov 04 07:19:27.490263 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.490245    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:27.494218 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.494194    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-tlbrh\""
Nov 04 07:19:27.500684 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.500660    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"]
Nov 04 07:19:27.564472 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.564440    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9\" (UID: \"9c102465-8050-4a38-8d87-a8149acf7f9f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:27.564641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.564480    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9\" (UID: \"9c102465-8050-4a38-8d87-a8149acf7f9f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:27.564641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.564521    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqdkv\" (UniqueName: \"kubernetes.io/projected/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-kube-api-access-nqdkv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd\" (UID: \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:27.564765 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.564651    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd\" (UID: \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:27.564833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.564779    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd\" (UID: \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:27.564909 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.564892    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9\" (UID: \"9c102465-8050-4a38-8d87-a8149acf7f9f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:27.564965 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.564889    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6x5d\" (UniqueName: \"kubernetes.io/projected/9c102465-8050-4a38-8d87-a8149acf7f9f-kube-api-access-r6x5d\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9\" (UID: \"9c102465-8050-4a38-8d87-a8149acf7f9f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:27.565020 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.564961    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9\" (UID: \"9c102465-8050-4a38-8d87-a8149acf7f9f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:27.577200 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.577173    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6x5d\" (UniqueName: \"kubernetes.io/projected/9c102465-8050-4a38-8d87-a8149acf7f9f-kube-api-access-r6x5d\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9\" (UID: \"9c102465-8050-4a38-8d87-a8149acf7f9f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:27.582229 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.582203    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"]
Nov 04 07:19:27.586095 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.586068    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:27.590294 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.590273    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-t9mcj\""
Nov 04 07:19:27.595955 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.595932    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"]
Nov 04 07:19:27.666387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.666280    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52xxc\" (UniqueName: \"kubernetes.io/projected/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-kube-api-access-52xxc\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w\" (UID: \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:27.666387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.666322    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w\" (UID: \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:27.666387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.666347    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w\" (UID: \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:27.666652 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.666519    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqdkv\" (UniqueName: \"kubernetes.io/projected/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-kube-api-access-nqdkv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd\" (UID: \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:27.666652 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.666578    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd\" (UID: \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:27.666652 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.666620    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd\" (UID: \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:27.667009 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.666984    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd\" (UID: \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:27.667119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.667036    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd\" (UID: \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:27.679743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.679708    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqdkv\" (UniqueName: \"kubernetes.io/projected/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-kube-api-access-nqdkv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd\" (UID: \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:27.689243 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.687435    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"]
Nov 04 07:19:27.691484 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.691465    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:27.695469 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.695450    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-w5sgt\""
Nov 04 07:19:27.702131 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.702108    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"]
Nov 04 07:19:27.709139 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.709120    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:27.767311 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.767282    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52xxc\" (UniqueName: \"kubernetes.io/projected/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-kube-api-access-52xxc\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w\" (UID: \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:27.767458 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.767332    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w\" (UID: \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:27.767458 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.767382    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w\" (UID: \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:27.767458 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.767426    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77\" (UID: \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:27.767621 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.767477    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77\" (UID: \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:27.767621 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.767535    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zcmd\" (UniqueName: \"kubernetes.io/projected/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-kube-api-access-5zcmd\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77\" (UID: \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:27.768170 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.768118    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w\" (UID: \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:27.768278 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.768167    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w\" (UID: \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:27.781328 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.781281    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52xxc\" (UniqueName: \"kubernetes.io/projected/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-kube-api-access-52xxc\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w\" (UID: \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:27.807934 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.807910    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:27.869114 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.868985    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77\" (UID: \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:27.869114 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.869049    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77\" (UID: \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:27.869114 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.869105    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zcmd\" (UniqueName: \"kubernetes.io/projected/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-kube-api-access-5zcmd\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77\" (UID: \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:27.870021 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.869877    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77\" (UID: \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:27.870428 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.870139    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77\" (UID: \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:27.881956 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.881928    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zcmd\" (UniqueName: \"kubernetes.io/projected/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-kube-api-access-5zcmd\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77\" (UID: \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:27.905454 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.905428    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:27.992237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:27.992156    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"]
Nov 04 07:19:27.995162 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:19:27.995133    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c102465_8050_4a38_8d87_a8149acf7f9f.slice/crio-f32baeeb0bf47e0e52a8515595343c70ae59d4aa182e84145a3005bc0aa254b7 WatchSource:0}: Error finding container f32baeeb0bf47e0e52a8515595343c70ae59d4aa182e84145a3005bc0aa254b7: Status 404 returned error can't find the container with id f32baeeb0bf47e0e52a8515595343c70ae59d4aa182e84145a3005bc0aa254b7
Nov 04 07:19:28.012933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.012908    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:28.105432 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.105407    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"]
Nov 04 07:19:28.108444 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:19:28.108415    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd79e96_7737_471d_8cc7_02ad84cfbe5f.slice/crio-d2e4014ee05fe66b918b256014d6739e96312b69a3be27599b21b075d612164f WatchSource:0}: Error finding container d2e4014ee05fe66b918b256014d6739e96312b69a3be27599b21b075d612164f: Status 404 returned error can't find the container with id d2e4014ee05fe66b918b256014d6739e96312b69a3be27599b21b075d612164f
Nov 04 07:19:28.181918 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.181845    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"]
Nov 04 07:19:28.185334 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:19:28.185255    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c2f2af_1ecd_41f8_b849_2b44d37a5d9c.slice/crio-4f4e8f4829a6fb7ef2a1fcef39b6b1d48e7ea010f3c5c67ce7149511ab2c0209 WatchSource:0}: Error finding container 4f4e8f4829a6fb7ef2a1fcef39b6b1d48e7ea010f3c5c67ce7149511ab2c0209: Status 404 returned error can't find the container with id 4f4e8f4829a6fb7ef2a1fcef39b6b1d48e7ea010f3c5c67ce7149511ab2c0209
Nov 04 07:19:28.296724 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.296689    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"]
Nov 04 07:19:28.302647 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:19:28.302618    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3dab776_b13b_4e40_bcb3_f5d20d4d49a9.slice/crio-c5be9b076dac5976bb4b4eca7387168c2e1dbd3c4857e0e802b000e5a3411dc5 WatchSource:0}: Error finding container c5be9b076dac5976bb4b4eca7387168c2e1dbd3c4857e0e802b000e5a3411dc5: Status 404 returned error can't find the container with id c5be9b076dac5976bb4b4eca7387168c2e1dbd3c4857e0e802b000e5a3411dc5
Nov 04 07:19:28.727171 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.724896    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"]
Nov 04 07:19:28.729887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.729171    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:28.734244 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.734197    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-2m67d\""
Nov 04 07:19:28.743028 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.742965    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"]
Nov 04 07:19:28.777914 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.777879    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg\" (UID: \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:28.778069 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.777984    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdphl\" (UniqueName: \"kubernetes.io/projected/a9c9d249-e95c-493c-aca6-710cec5bf1a5-kube-api-access-rdphl\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg\" (UID: \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:28.778069 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.778030    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg\" (UID: \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:28.833415 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.833310    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77" event={"ID":"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9","Type":"ContainerStarted","Data":"c5be9b076dac5976bb4b4eca7387168c2e1dbd3c4857e0e802b000e5a3411dc5"}
Nov 04 07:19:28.835376 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.835294    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9" event={"ID":"9c102465-8050-4a38-8d87-a8149acf7f9f","Type":"ContainerStarted","Data":"f32baeeb0bf47e0e52a8515595343c70ae59d4aa182e84145a3005bc0aa254b7"}
Nov 04 07:19:28.837946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.837918    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w" event={"ID":"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c","Type":"ContainerStarted","Data":"4f4e8f4829a6fb7ef2a1fcef39b6b1d48e7ea010f3c5c67ce7149511ab2c0209"}
Nov 04 07:19:28.839548 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.839518    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd" event={"ID":"ebd79e96-7737-471d-8cc7-02ad84cfbe5f","Type":"ContainerStarted","Data":"d2e4014ee05fe66b918b256014d6739e96312b69a3be27599b21b075d612164f"}
Nov 04 07:19:28.879224 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.879189    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg\" (UID: \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:28.879676 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.879295    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdphl\" (UniqueName: \"kubernetes.io/projected/a9c9d249-e95c-493c-aca6-710cec5bf1a5-kube-api-access-rdphl\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg\" (UID: \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:28.879676 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.879341    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg\" (UID: \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:28.879911 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.879888    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg\" (UID: \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:28.880158 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.880137    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg\" (UID: \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:28.901676 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:28.901654    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdphl\" (UniqueName: \"kubernetes.io/projected/a9c9d249-e95c-493c-aca6-710cec5bf1a5-kube-api-access-rdphl\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg\" (UID: \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:29.065644 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.065390    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:29.259692 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.259097    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"]
Nov 04 07:19:29.263175 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.263151    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.280196 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.280153    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 07:19:29.280590 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.280569    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 07:19:29.282141 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.281341    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"]
Nov 04 07:19:29.282141 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.281484    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 07:19:29.284196 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.284169    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2rn5b\""
Nov 04 07:19:29.385344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.384808    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.385344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.384853    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ba40135-db97-4238-8bb8-f1fd581a1615-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.385344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.384883    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fmvw\" (UniqueName: \"kubernetes.io/projected/7ba40135-db97-4238-8bb8-f1fd581a1615-kube-api-access-6fmvw\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.385344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.384931    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ba40135-db97-4238-8bb8-f1fd581a1615-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.385344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.384966    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.385344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.384997    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.385344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.385084    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.385344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.385112    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.385344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.385139    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ba40135-db97-4238-8bb8-f1fd581a1615-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.418915 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.418885    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"]
Nov 04 07:19:29.424974 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:19:29.424945    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9c9d249_e95c_493c_aca6_710cec5bf1a5.slice/crio-6c17682f134a0da803442c2d948bbe5a6521f9853759b6b3da69f880a8916d8d WatchSource:0}: Error finding container 6c17682f134a0da803442c2d948bbe5a6521f9853759b6b3da69f880a8916d8d: Status 404 returned error can't find the container with id 6c17682f134a0da803442c2d948bbe5a6521f9853759b6b3da69f880a8916d8d
Nov 04 07:19:29.486002 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.485963    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.486162 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.486017    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.486162 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.486075    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ba40135-db97-4238-8bb8-f1fd581a1615-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.486162 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.486113    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.486162 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.486144    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ba40135-db97-4238-8bb8-f1fd581a1615-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.486345 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.486194    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fmvw\" (UniqueName: \"kubernetes.io/projected/7ba40135-db97-4238-8bb8-f1fd581a1615-kube-api-access-6fmvw\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.486345 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.486247    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ba40135-db97-4238-8bb8-f1fd581a1615-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.486345 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.486314    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.486492 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.486353    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.486951 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.486927    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.487273 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.487195    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.488774 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.487645    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.488774 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.487983    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.488774 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.488502    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ba40135-db97-4238-8bb8-f1fd581a1615-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.490289 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.490248    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ba40135-db97-4238-8bb8-f1fd581a1615-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.490607 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.490555    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ba40135-db97-4238-8bb8-f1fd581a1615-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.500479 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.500441    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ba40135-db97-4238-8bb8-f1fd581a1615-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.505415 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.505343    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fmvw\" (UniqueName: \"kubernetes.io/projected/7ba40135-db97-4238-8bb8-f1fd581a1615-kube-api-access-6fmvw\") pod \"maas-default-gateway-openshift-default-74854d8ccf-xgjgh\" (UID: \"7ba40135-db97-4238-8bb8-f1fd581a1615\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.583904 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.583830    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:29.848012 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.847918    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg" event={"ID":"a9c9d249-e95c-493c-aca6-710cec5bf1a5","Type":"ContainerStarted","Data":"6c17682f134a0da803442c2d948bbe5a6521f9853759b6b3da69f880a8916d8d"}
Nov 04 07:19:29.940695 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:29.940662    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"]
Nov 04 07:19:29.945149 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:19:29.945117    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ba40135_db97_4238_8bb8_f1fd581a1615.slice/crio-6cbb17a0ec301e06b7722727b70de3033f131f988424984365b8ccbc5df9df21 WatchSource:0}: Error finding container 6cbb17a0ec301e06b7722727b70de3033f131f988424984365b8ccbc5df9df21: Status 404 returned error can't find the container with id 6cbb17a0ec301e06b7722727b70de3033f131f988424984365b8ccbc5df9df21
Nov 04 07:19:30.860355 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:30.860246    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh" event={"ID":"7ba40135-db97-4238-8bb8-f1fd581a1615","Type":"ContainerStarted","Data":"6cbb17a0ec301e06b7722727b70de3033f131f988424984365b8ccbc5df9df21"}
Nov 04 07:19:36.187036 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:19:36.186989    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:19:36.600847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:36.600780    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:19:36.600984 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:36.600873    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:19:36.600984 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:36.600946    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:19:36.901410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:36.901040    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh" event={"ID":"7ba40135-db97-4238-8bb8-f1fd581a1615","Type":"ContainerStarted","Data":"a0302d10b93d4ea4d71ac43b26329a2c6e6984c5fd0019630cd38d1a2bc3fcac"}
Nov 04 07:19:36.929497 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:36.929441    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh" podStartSLOduration=1.279580699 podStartE2EDuration="7.929422879s" totalImagesPullingTime="6.64984218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:19:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:19:36.9273228 +0000 UTC m=+198.949538060" watchObservedRunningTime="2026-11-04 07:19:36.929422879 +0000 UTC m=+198.951638152"
Nov 04 07:19:37.584492 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:37.584434    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:37.590253 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:37.590229    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:37.909502 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:37.909424    5626 generic.go:356] "Generic (PLEG): container finished" podID="8b2301ed-5209-45f7-8262-51fb2f80ac33" containerID="1e2f48ad4ac299fa2df5be5a32e26574ad9a9506ce95b53584de05c5f9aee014" exitCode=0
Nov 04 07:19:37.909502 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:37.909499    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk" event={"ID":"8b2301ed-5209-45f7-8262-51fb2f80ac33","Type":"ContainerDied","Data":"1e2f48ad4ac299fa2df5be5a32e26574ad9a9506ce95b53584de05c5f9aee014"}
Nov 04 07:19:37.913946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:37.913916    5626 generic.go:356] "Generic (PLEG): container finished" podID="a9c9d249-e95c-493c-aca6-710cec5bf1a5" containerID="cd4f75ec427f3f2af17bb10db806d6e2542d8edd052f229df1b82a16a43b842b" exitCode=0
Nov 04 07:19:37.915417 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:37.915394    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg" event={"ID":"a9c9d249-e95c-493c-aca6-710cec5bf1a5","Type":"ContainerDied","Data":"cd4f75ec427f3f2af17bb10db806d6e2542d8edd052f229df1b82a16a43b842b"}
Nov 04 07:19:37.915498 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:37.915445    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:37.916554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:37.916515    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:19:38.925831 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:38.925792    5626 generic.go:356] "Generic (PLEG): container finished" podID="8b2301ed-5209-45f7-8262-51fb2f80ac33" containerID="3864e8e92e7e02bc7d38fd97cdb9041301a4e0e0b9465c3d8e7bc35b6365e29c" exitCode=0
Nov 04 07:19:38.926275 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:38.925875    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk" event={"ID":"8b2301ed-5209-45f7-8262-51fb2f80ac33","Type":"ContainerDied","Data":"3864e8e92e7e02bc7d38fd97cdb9041301a4e0e0b9465c3d8e7bc35b6365e29c"}
Nov 04 07:19:40.939570 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:40.939535    5626 generic.go:356] "Generic (PLEG): container finished" podID="9c102465-8050-4a38-8d87-a8149acf7f9f" containerID="76b24291a4e9745340a66955b97157bd43d6ca21b888fd1a4f4bfae0593f862b" exitCode=0
Nov 04 07:19:40.940073 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:40.939629    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9" event={"ID":"9c102465-8050-4a38-8d87-a8149acf7f9f","Type":"ContainerDied","Data":"76b24291a4e9745340a66955b97157bd43d6ca21b888fd1a4f4bfae0593f862b"}
Nov 04 07:19:40.942432 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:40.941557    5626 generic.go:356] "Generic (PLEG): container finished" podID="40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" containerID="a6612c0cd17504902960b5496e82ae528650b6f7dd613088c3cce70094be73c5" exitCode=0
Nov 04 07:19:40.942432 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:40.942051    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w" event={"ID":"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c","Type":"ContainerDied","Data":"a6612c0cd17504902960b5496e82ae528650b6f7dd613088c3cce70094be73c5"}
Nov 04 07:19:40.943677 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:40.943658    5626 generic.go:356] "Generic (PLEG): container finished" podID="ebd79e96-7737-471d-8cc7-02ad84cfbe5f" containerID="7909136e7f012a0b8ab95c8240977c0395d7b7857ae9ce62dceb9d4b37800493" exitCode=0
Nov 04 07:19:40.943776 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:40.943753    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd" event={"ID":"ebd79e96-7737-471d-8cc7-02ad84cfbe5f","Type":"ContainerDied","Data":"7909136e7f012a0b8ab95c8240977c0395d7b7857ae9ce62dceb9d4b37800493"}
Nov 04 07:19:40.946956 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:40.946933    5626 generic.go:356] "Generic (PLEG): container finished" podID="c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" containerID="3ffbb597c004a4e8f496e7b43248ec64ae62b35f8edbf50c70a1f4356f0103cd" exitCode=0
Nov 04 07:19:40.947054 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:40.946991    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77" event={"ID":"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9","Type":"ContainerDied","Data":"3ffbb597c004a4e8f496e7b43248ec64ae62b35f8edbf50c70a1f4356f0103cd"}
Nov 04 07:19:42.959290 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:42.959211    5626 generic.go:356] "Generic (PLEG): container finished" podID="c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" containerID="2fefd62d2b8267af06040268a59f1e8e4485d99a91761b2b917631df01f66926" exitCode=0
Nov 04 07:19:42.959723 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:42.959566    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77" event={"ID":"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9","Type":"ContainerDied","Data":"2fefd62d2b8267af06040268a59f1e8e4485d99a91761b2b917631df01f66926"}
Nov 04 07:19:42.961755 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:42.961667    5626 generic.go:356] "Generic (PLEG): container finished" podID="9c102465-8050-4a38-8d87-a8149acf7f9f" containerID="60aa84dbbd464b18230268a8ab533b8513f5825cb642e9c29259e5b95517871f" exitCode=0
Nov 04 07:19:42.961862 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:42.961752    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9" event={"ID":"9c102465-8050-4a38-8d87-a8149acf7f9f","Type":"ContainerDied","Data":"60aa84dbbd464b18230268a8ab533b8513f5825cb642e9c29259e5b95517871f"}
Nov 04 07:19:42.967025 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:42.965613    5626 generic.go:356] "Generic (PLEG): container finished" podID="40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" containerID="df4087c4a1648b44c536c1f20ffc2bd9a48bcaebba3fcdee9a60fe54ae5d120c" exitCode=0
Nov 04 07:19:42.967025 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:42.965716    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w" event={"ID":"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c","Type":"ContainerDied","Data":"df4087c4a1648b44c536c1f20ffc2bd9a48bcaebba3fcdee9a60fe54ae5d120c"}
Nov 04 07:19:42.968969 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:42.968946    5626 generic.go:356] "Generic (PLEG): container finished" podID="a9c9d249-e95c-493c-aca6-710cec5bf1a5" containerID="98e9c5805407d56160054f6bf75a3c15a3559f9b636a64a0f178a222e4c74672" exitCode=0
Nov 04 07:19:42.969213 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:42.969197    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg" event={"ID":"a9c9d249-e95c-493c-aca6-710cec5bf1a5","Type":"ContainerDied","Data":"98e9c5805407d56160054f6bf75a3c15a3559f9b636a64a0f178a222e4c74672"}
Nov 04 07:19:42.971815 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:42.971732    5626 generic.go:356] "Generic (PLEG): container finished" podID="ebd79e96-7737-471d-8cc7-02ad84cfbe5f" containerID="e8abd5401bd74238f8917e7c10791ca142336b288060b515c68333fc8c4a7646" exitCode=0
Nov 04 07:19:42.971815 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:42.971775    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd" event={"ID":"ebd79e96-7737-471d-8cc7-02ad84cfbe5f","Type":"ContainerDied","Data":"e8abd5401bd74238f8917e7c10791ca142336b288060b515c68333fc8c4a7646"}
Nov 04 07:19:50.029984 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.029946    5626 generic.go:356] "Generic (PLEG): container finished" podID="8b2301ed-5209-45f7-8262-51fb2f80ac33" containerID="d7dc1408083fcce37feb143859162ea92ca01c445c3a7a1f568ed0134ae53611" exitCode=0
Nov 04 07:19:50.030486 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.030035    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk" event={"ID":"8b2301ed-5209-45f7-8262-51fb2f80ac33","Type":"ContainerDied","Data":"d7dc1408083fcce37feb143859162ea92ca01c445c3a7a1f568ed0134ae53611"}
Nov 04 07:19:50.429642 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.428303    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm"]
Nov 04 07:19:50.430662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.430639    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm"
Nov 04 07:19:50.435384 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.435163    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 04 07:19:50.435499 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.435434    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-tm5bw\""
Nov 04 07:19:50.435499 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.435486    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 04 07:19:50.444129 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.444104    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm"]
Nov 04 07:19:50.607293 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.607259    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdtcv\" (UniqueName: \"kubernetes.io/projected/5c70f97d-8ee2-4adb-9d82-6cf949ab02fd-kube-api-access-qdtcv\") pod \"jobset-operator-674cd59dc5-lrplm\" (UID: \"5c70f97d-8ee2-4adb-9d82-6cf949ab02fd\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm"
Nov 04 07:19:50.607483 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.607400    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c70f97d-8ee2-4adb-9d82-6cf949ab02fd-tmp\") pod \"jobset-operator-674cd59dc5-lrplm\" (UID: \"5c70f97d-8ee2-4adb-9d82-6cf949ab02fd\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm"
Nov 04 07:19:50.708748 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.708719    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdtcv\" (UniqueName: \"kubernetes.io/projected/5c70f97d-8ee2-4adb-9d82-6cf949ab02fd-kube-api-access-qdtcv\") pod \"jobset-operator-674cd59dc5-lrplm\" (UID: \"5c70f97d-8ee2-4adb-9d82-6cf949ab02fd\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm"
Nov 04 07:19:50.708849 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.708820    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c70f97d-8ee2-4adb-9d82-6cf949ab02fd-tmp\") pod \"jobset-operator-674cd59dc5-lrplm\" (UID: \"5c70f97d-8ee2-4adb-9d82-6cf949ab02fd\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm"
Nov 04 07:19:50.709597 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.709551    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c70f97d-8ee2-4adb-9d82-6cf949ab02fd-tmp\") pod \"jobset-operator-674cd59dc5-lrplm\" (UID: \"5c70f97d-8ee2-4adb-9d82-6cf949ab02fd\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm"
Nov 04 07:19:50.722656 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.722607    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdtcv\" (UniqueName: \"kubernetes.io/projected/5c70f97d-8ee2-4adb-9d82-6cf949ab02fd-kube-api-access-qdtcv\") pod \"jobset-operator-674cd59dc5-lrplm\" (UID: \"5c70f97d-8ee2-4adb-9d82-6cf949ab02fd\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm"
Nov 04 07:19:50.755653 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:50.755443    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm"
Nov 04 07:19:51.038668 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.038637    5626 generic.go:356] "Generic (PLEG): container finished" podID="c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" containerID="a88e6e216e4e276dac87a8fd1429e494641b8c6db750a8f978c4d84a26c3675f" exitCode=0
Nov 04 07:19:51.039037 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.038714    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77" event={"ID":"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9","Type":"ContainerDied","Data":"a88e6e216e4e276dac87a8fd1429e494641b8c6db750a8f978c4d84a26c3675f"}
Nov 04 07:19:51.041309 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.041279    5626 generic.go:356] "Generic (PLEG): container finished" podID="9c102465-8050-4a38-8d87-a8149acf7f9f" containerID="4ab9db433e484670ab7cc3a4ef4efe36cf455bfd0f08a03b65481f0355534808" exitCode=0
Nov 04 07:19:51.041437 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.041387    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9" event={"ID":"9c102465-8050-4a38-8d87-a8149acf7f9f","Type":"ContainerDied","Data":"4ab9db433e484670ab7cc3a4ef4efe36cf455bfd0f08a03b65481f0355534808"}
Nov 04 07:19:51.044650 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.044621    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w" event={"ID":"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c","Type":"ContainerStarted","Data":"a55a8a269c2d3294801c62c5c04bea92bec5ab02077a65032751f3a01aca08bc"}
Nov 04 07:19:51.048269 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.048241    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg" event={"ID":"a9c9d249-e95c-493c-aca6-710cec5bf1a5","Type":"ContainerStarted","Data":"928e87c9b2c27eb46ce219079c0d08737bc2ab7dba53ee8391860ffdc7b99547"}
Nov 04 07:19:51.048535 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.048513    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm"]
Nov 04 07:19:51.051541 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:19:51.050161    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c70f97d_8ee2_4adb_9d82_6cf949ab02fd.slice/crio-84fa7530342266b1f90ac889230c3ba990ed3a1b0c24022cfc71ea2499e7fb3e WatchSource:0}: Error finding container 84fa7530342266b1f90ac889230c3ba990ed3a1b0c24022cfc71ea2499e7fb3e: Status 404 returned error can't find the container with id 84fa7530342266b1f90ac889230c3ba990ed3a1b0c24022cfc71ea2499e7fb3e
Nov 04 07:19:51.052182 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.052162    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd" event={"ID":"ebd79e96-7737-471d-8cc7-02ad84cfbe5f","Type":"ContainerStarted","Data":"91540690cb70ce57772e83e0a3386a3cb7e537ca51a30410eb2325bb77482d8e"}
Nov 04 07:19:51.088257 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.088206    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg" podStartSLOduration=2.9895115150000002 podStartE2EDuration="23.088190803s" totalImagesPullingTime="20.098679288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:19:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:19:51.087427151 +0000 UTC m=+213.109642411" watchObservedRunningTime="2026-11-04 07:19:51.088190803 +0000 UTC m=+213.110406064"
Nov 04 07:19:51.131673 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.131628    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w" podStartSLOduration=2.617312176 podStartE2EDuration="24.131613985s" totalImagesPullingTime="21.514301809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:19:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:19:51.130923159 +0000 UTC m=+213.153138661" watchObservedRunningTime="2026-11-04 07:19:51.131613985 +0000 UTC m=+213.153829242"
Nov 04 07:19:51.157882 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.157812    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd" podStartSLOduration=2.03379046 podStartE2EDuration="24.157791921s" totalImagesPullingTime="22.124001461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:19:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:19:51.156473547 +0000 UTC m=+213.178688808" watchObservedRunningTime="2026-11-04 07:19:51.157791921 +0000 UTC m=+213.180007182"
Nov 04 07:19:51.233276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.233220    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:51.416725 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.416683    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b2301ed-5209-45f7-8262-51fb2f80ac33-kube-api-access-j99r4\" (UniqueName: \"kubernetes.io/projected/8b2301ed-5209-45f7-8262-51fb2f80ac33-kube-api-access-j99r4\") pod \"8b2301ed-5209-45f7-8262-51fb2f80ac33\" (UID: \"8b2301ed-5209-45f7-8262-51fb2f80ac33\") "
Nov 04 07:19:51.416944 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.416749    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-bundle\") pod \"8b2301ed-5209-45f7-8262-51fb2f80ac33\" (UID: \"8b2301ed-5209-45f7-8262-51fb2f80ac33\") "
Nov 04 07:19:51.416944 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.416856    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-util\" (UniqueName: \"kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-util\") pod \"8b2301ed-5209-45f7-8262-51fb2f80ac33\" (UID: \"8b2301ed-5209-45f7-8262-51fb2f80ac33\") "
Nov 04 07:19:51.417275 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.417251    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-bundle" pod "8b2301ed-5209-45f7-8262-51fb2f80ac33" (UID: "8b2301ed-5209-45f7-8262-51fb2f80ac33"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:19:51.418932 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.418891    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b2301ed-5209-45f7-8262-51fb2f80ac33-kube-api-access-j99r4" pod "8b2301ed-5209-45f7-8262-51fb2f80ac33" (UID: "8b2301ed-5209-45f7-8262-51fb2f80ac33"). InnerVolumeSpecName "kube-api-access-j99r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:19:51.421779 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.421748    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-util" pod "8b2301ed-5209-45f7-8262-51fb2f80ac33" (UID: "8b2301ed-5209-45f7-8262-51fb2f80ac33"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:19:51.518093 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.518018    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j99r4\" (UniqueName: \"kubernetes.io/projected/8b2301ed-5209-45f7-8262-51fb2f80ac33-kube-api-access-j99r4\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:51.518093 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.518048    5626 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-bundle\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:51.518093 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:51.518058    5626 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b2301ed-5209-45f7-8262-51fb2f80ac33-util\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:52.063380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.063318    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk" event={"ID":"8b2301ed-5209-45f7-8262-51fb2f80ac33","Type":"ContainerDied","Data":"50af61bdd20ae7bb1155c454322acbb55f3c219655ed0b70517975be8963a0d4"}
Nov 04 07:19:52.063819 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.063432    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50af61bdd20ae7bb1155c454322acbb55f3c219655ed0b70517975be8963a0d4"
Nov 04 07:19:52.063819 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.063511    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7159xnk"
Nov 04 07:19:52.067771 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.067729    5626 generic.go:356] "Generic (PLEG): container finished" podID="a9c9d249-e95c-493c-aca6-710cec5bf1a5" containerID="928e87c9b2c27eb46ce219079c0d08737bc2ab7dba53ee8391860ffdc7b99547" exitCode=0
Nov 04 07:19:52.067897 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.067810    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg" event={"ID":"a9c9d249-e95c-493c-aca6-710cec5bf1a5","Type":"ContainerDied","Data":"928e87c9b2c27eb46ce219079c0d08737bc2ab7dba53ee8391860ffdc7b99547"}
Nov 04 07:19:52.070562 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.070526    5626 generic.go:356] "Generic (PLEG): container finished" podID="ebd79e96-7737-471d-8cc7-02ad84cfbe5f" containerID="91540690cb70ce57772e83e0a3386a3cb7e537ca51a30410eb2325bb77482d8e" exitCode=0
Nov 04 07:19:52.070685 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.070622    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd" event={"ID":"ebd79e96-7737-471d-8cc7-02ad84cfbe5f","Type":"ContainerDied","Data":"91540690cb70ce57772e83e0a3386a3cb7e537ca51a30410eb2325bb77482d8e"}
Nov 04 07:19:52.072639 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.072616    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm" event={"ID":"5c70f97d-8ee2-4adb-9d82-6cf949ab02fd","Type":"ContainerStarted","Data":"84fa7530342266b1f90ac889230c3ba990ed3a1b0c24022cfc71ea2499e7fb3e"}
Nov 04 07:19:52.075580 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.075494    5626 generic.go:356] "Generic (PLEG): container finished" podID="40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" containerID="a55a8a269c2d3294801c62c5c04bea92bec5ab02077a65032751f3a01aca08bc" exitCode=0
Nov 04 07:19:52.075580 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.075542    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w" event={"ID":"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c","Type":"ContainerDied","Data":"a55a8a269c2d3294801c62c5c04bea92bec5ab02077a65032751f3a01aca08bc"}
Nov 04 07:19:52.233247 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.232781    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:52.290823 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.289143    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:52.424228 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.424148    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-bundle\") pod \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\" (UID: \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\") "
Nov 04 07:19:52.424390 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.424254    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-util\" (UniqueName: \"kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-util\") pod \"9c102465-8050-4a38-8d87-a8149acf7f9f\" (UID: \"9c102465-8050-4a38-8d87-a8149acf7f9f\") "
Nov 04 07:19:52.424390 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.424280    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c102465-8050-4a38-8d87-a8149acf7f9f-kube-api-access-r6x5d\" (UniqueName: \"kubernetes.io/projected/9c102465-8050-4a38-8d87-a8149acf7f9f-kube-api-access-r6x5d\") pod \"9c102465-8050-4a38-8d87-a8149acf7f9f\" (UID: \"9c102465-8050-4a38-8d87-a8149acf7f9f\") "
Nov 04 07:19:52.424390 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.424301    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-bundle\" (UniqueName: \"kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-bundle\") pod \"9c102465-8050-4a38-8d87-a8149acf7f9f\" (UID: \"9c102465-8050-4a38-8d87-a8149acf7f9f\") "
Nov 04 07:19:52.424390 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.424333    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-kube-api-access-5zcmd\" (UniqueName: \"kubernetes.io/projected/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-kube-api-access-5zcmd\") pod \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\" (UID: \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\") "
Nov 04 07:19:52.424390 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.424356    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-util\" (UniqueName: \"kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-util\") pod \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\" (UID: \"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9\") "
Nov 04 07:19:52.425492 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.425399    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-bundle" pod "9c102465-8050-4a38-8d87-a8149acf7f9f" (UID: "9c102465-8050-4a38-8d87-a8149acf7f9f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:19:52.425704 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.425667    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-bundle" pod "c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" (UID: "c3dab776-b13b-4e40-bcb3-f5d20d4d49a9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:19:52.427455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.427416    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c102465-8050-4a38-8d87-a8149acf7f9f-kube-api-access-r6x5d" pod "9c102465-8050-4a38-8d87-a8149acf7f9f" (UID: "9c102465-8050-4a38-8d87-a8149acf7f9f"). InnerVolumeSpecName "kube-api-access-r6x5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:19:52.427759 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.427727    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-kube-api-access-5zcmd" pod "c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" (UID: "c3dab776-b13b-4e40-bcb3-f5d20d4d49a9"). InnerVolumeSpecName "kube-api-access-5zcmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:19:52.432157 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.432099    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-util" pod "c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" (UID: "c3dab776-b13b-4e40-bcb3-f5d20d4d49a9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:19:52.432157 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.432095    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-util" pod "9c102465-8050-4a38-8d87-a8149acf7f9f" (UID: "9c102465-8050-4a38-8d87-a8149acf7f9f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:19:52.525332 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.525296    5626 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-util\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:52.525530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.525338    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6x5d\" (UniqueName: \"kubernetes.io/projected/9c102465-8050-4a38-8d87-a8149acf7f9f-kube-api-access-r6x5d\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:52.525530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.525357    5626 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9c102465-8050-4a38-8d87-a8149acf7f9f-bundle\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:52.525530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.525394    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zcmd\" (UniqueName: \"kubernetes.io/projected/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-kube-api-access-5zcmd\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:52.525530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.525409    5626 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-util\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:52.525530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:52.525425    5626 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3dab776-b13b-4e40-bcb3-f5d20d4d49a9-bundle\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:53.083017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.082947    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9"
Nov 04 07:19:53.083017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.082965    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crj5z9" event={"ID":"9c102465-8050-4a38-8d87-a8149acf7f9f","Type":"ContainerDied","Data":"f32baeeb0bf47e0e52a8515595343c70ae59d4aa182e84145a3005bc0aa254b7"}
Nov 04 07:19:53.083534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.083019    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f32baeeb0bf47e0e52a8515595343c70ae59d4aa182e84145a3005bc0aa254b7"
Nov 04 07:19:53.085314 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.085287    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77" event={"ID":"c3dab776-b13b-4e40-bcb3-f5d20d4d49a9","Type":"ContainerDied","Data":"c5be9b076dac5976bb4b4eca7387168c2e1dbd3c4857e0e802b000e5a3411dc5"}
Nov 04 07:19:53.085459 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.085324    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5be9b076dac5976bb4b4eca7387168c2e1dbd3c4857e0e802b000e5a3411dc5"
Nov 04 07:19:53.085459 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.085344    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6cn77"
Nov 04 07:19:53.246341 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.246314    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:53.275846 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.275734    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:53.283878 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.283853    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:53.433549 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.433478    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-util\" (UniqueName: \"kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-util\") pod \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\" (UID: \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\") "
Nov 04 07:19:53.433549 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.433545    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-kube-api-access-52xxc\" (UniqueName: \"kubernetes.io/projected/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-kube-api-access-52xxc\") pod \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\" (UID: \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\") "
Nov 04 07:19:53.433768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.433589    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-util\" (UniqueName: \"kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-util\") pod \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\" (UID: \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\") "
Nov 04 07:19:53.433768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.433623    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9c9d249-e95c-493c-aca6-710cec5bf1a5-kube-api-access-rdphl\" (UniqueName: \"kubernetes.io/projected/a9c9d249-e95c-493c-aca6-710cec5bf1a5-kube-api-access-rdphl\") pod \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\" (UID: \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\") "
Nov 04 07:19:53.433768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.433680    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-util\" (UniqueName: \"kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-util\") pod \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\" (UID: \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\") "
Nov 04 07:19:53.433768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.433713    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-bundle\") pod \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\" (UID: \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\") "
Nov 04 07:19:53.433768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.433739    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-bundle\") pod \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\" (UID: \"a9c9d249-e95c-493c-aca6-710cec5bf1a5\") "
Nov 04 07:19:53.434029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.433799    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-bundle\" (UniqueName: \"kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-bundle\") pod \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\" (UID: \"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c\") "
Nov 04 07:19:53.434029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.433863    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-kube-api-access-nqdkv\" (UniqueName: \"kubernetes.io/projected/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-kube-api-access-nqdkv\") pod \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\" (UID: \"ebd79e96-7737-471d-8cc7-02ad84cfbe5f\") "
Nov 04 07:19:53.434805 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.434759    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-bundle" pod "40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" (UID: "40c2f2af-1ecd-41f8-b849-2b44d37a5d9c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:19:53.434805 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.434790    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-bundle" pod "a9c9d249-e95c-493c-aca6-710cec5bf1a5" (UID: "a9c9d249-e95c-493c-aca6-710cec5bf1a5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:19:53.434805 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.434799    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-bundle" pod "ebd79e96-7737-471d-8cc7-02ad84cfbe5f" (UID: "ebd79e96-7737-471d-8cc7-02ad84cfbe5f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:19:53.436349 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.436319    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-kube-api-access-52xxc" pod "40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" (UID: "40c2f2af-1ecd-41f8-b849-2b44d37a5d9c"). InnerVolumeSpecName "kube-api-access-52xxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:19:53.437669 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.437642    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-kube-api-access-nqdkv" pod "ebd79e96-7737-471d-8cc7-02ad84cfbe5f" (UID: "ebd79e96-7737-471d-8cc7-02ad84cfbe5f"). InnerVolumeSpecName "kube-api-access-nqdkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:19:53.437896 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.437879    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9c9d249-e95c-493c-aca6-710cec5bf1a5-kube-api-access-rdphl" pod "a9c9d249-e95c-493c-aca6-710cec5bf1a5" (UID: "a9c9d249-e95c-493c-aca6-710cec5bf1a5"). InnerVolumeSpecName "kube-api-access-rdphl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:19:53.438308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.438288    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-util" pod "ebd79e96-7737-471d-8cc7-02ad84cfbe5f" (UID: "ebd79e96-7737-471d-8cc7-02ad84cfbe5f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:19:53.438619 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.438600    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-util" pod "a9c9d249-e95c-493c-aca6-710cec5bf1a5" (UID: "a9c9d249-e95c-493c-aca6-710cec5bf1a5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:19:53.439788 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.439760    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-util" pod "40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" (UID: "40c2f2af-1ecd-41f8-b849-2b44d37a5d9c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:19:53.535414 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.535378    5626 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-util\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:53.535414 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.535413    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52xxc\" (UniqueName: \"kubernetes.io/projected/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-kube-api-access-52xxc\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:53.535602 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.535424    5626 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-util\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:53.535602 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.535435    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdphl\" (UniqueName: \"kubernetes.io/projected/a9c9d249-e95c-493c-aca6-710cec5bf1a5-kube-api-access-rdphl\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:53.535602 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.535445    5626 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-util\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:53.535602 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.535453    5626 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-bundle\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:53.535602 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.535462    5626 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9c9d249-e95c-493c-aca6-710cec5bf1a5-bundle\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:53.535602 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.535469    5626 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40c2f2af-1ecd-41f8-b849-2b44d37a5d9c-bundle\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:53.535602 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:53.535477    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqdkv\" (UniqueName: \"kubernetes.io/projected/ebd79e96-7737-471d-8cc7-02ad84cfbe5f-kube-api-access-nqdkv\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:19:54.095409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:54.093031    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w"
Nov 04 07:19:54.095409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:54.093043    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zm45w" event={"ID":"40c2f2af-1ecd-41f8-b849-2b44d37a5d9c","Type":"ContainerDied","Data":"4f4e8f4829a6fb7ef2a1fcef39b6b1d48e7ea010f3c5c67ce7149511ab2c0209"}
Nov 04 07:19:54.095409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:54.093082    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f4e8f4829a6fb7ef2a1fcef39b6b1d48e7ea010f3c5c67ce7149511ab2c0209"
Nov 04 07:19:54.104418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:54.104390    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg" event={"ID":"a9c9d249-e95c-493c-aca6-710cec5bf1a5","Type":"ContainerDied","Data":"6c17682f134a0da803442c2d948bbe5a6521f9853759b6b3da69f880a8916d8d"}
Nov 04 07:19:54.104533 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:54.104444    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c17682f134a0da803442c2d948bbe5a6521f9853759b6b3da69f880a8916d8d"
Nov 04 07:19:54.104533 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:54.104396    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cqb2zg"
Nov 04 07:19:54.106578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:54.106553    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd"
Nov 04 07:19:54.106578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:54.106569    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7qt8hd" event={"ID":"ebd79e96-7737-471d-8cc7-02ad84cfbe5f","Type":"ContainerDied","Data":"d2e4014ee05fe66b918b256014d6739e96312b69a3be27599b21b075d612164f"}
Nov 04 07:19:54.106721 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:54.106603    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2e4014ee05fe66b918b256014d6739e96312b69a3be27599b21b075d612164f"
Nov 04 07:19:55.112390 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:55.112342    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm" event={"ID":"5c70f97d-8ee2-4adb-9d82-6cf949ab02fd","Type":"ContainerStarted","Data":"4fc9456c5d07e06ff48bc77bdc0a6fee25a9ec15cde87db67be1c0a2b80e9952"}
Nov 04 07:19:55.132398 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:55.132328    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm" podStartSLOduration=2.09646094 podStartE2EDuration="5.13231699s" totalImagesPullingTime="3.03585605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:19:55.130829501 +0000 UTC m=+217.153044768" watchObservedRunningTime="2026-11-04 07:19:55.13231699 +0000 UTC m=+217.154532250"
Nov 04 07:19:58.383795 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.383759    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"]
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384068    5626 cpu_manager.go:414] "containerMap: removing container" podUID="40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" containerName="util"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384082    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" containerName="util"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384095    5626 cpu_manager.go:414] "containerMap: removing container" podUID="a9c9d249-e95c-493c-aca6-710cec5bf1a5" containerName="pull"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384101    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9c9d249-e95c-493c-aca6-710cec5bf1a5" containerName="pull"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384107    5626 cpu_manager.go:414] "containerMap: removing container" podUID="c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" containerName="pull"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384112    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" containerName="pull"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384118    5626 cpu_manager.go:414] "containerMap: removing container" podUID="40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" containerName="pull"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384122    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" containerName="pull"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384131    5626 cpu_manager.go:414] "containerMap: removing container" podUID="a9c9d249-e95c-493c-aca6-710cec5bf1a5" containerName="extract"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384136    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9c9d249-e95c-493c-aca6-710cec5bf1a5" containerName="extract"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384148    5626 cpu_manager.go:414] "containerMap: removing container" podUID="9c102465-8050-4a38-8d87-a8149acf7f9f" containerName="extract"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384156    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c102465-8050-4a38-8d87-a8149acf7f9f" containerName="extract"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384171    5626 cpu_manager.go:414] "containerMap: removing container" podUID="a9c9d249-e95c-493c-aca6-710cec5bf1a5" containerName="util"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384177    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9c9d249-e95c-493c-aca6-710cec5bf1a5" containerName="util"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384183    5626 cpu_manager.go:414] "containerMap: removing container" podUID="8b2301ed-5209-45f7-8262-51fb2f80ac33" containerName="pull"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384187    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b2301ed-5209-45f7-8262-51fb2f80ac33" containerName="pull"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384195    5626 cpu_manager.go:414] "containerMap: removing container" podUID="ebd79e96-7737-471d-8cc7-02ad84cfbe5f" containerName="util"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384199    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebd79e96-7737-471d-8cc7-02ad84cfbe5f" containerName="util"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384206    5626 cpu_manager.go:414] "containerMap: removing container" podUID="c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" containerName="util"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384210    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" containerName="util"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384217    5626 cpu_manager.go:414] "containerMap: removing container" podUID="ebd79e96-7737-471d-8cc7-02ad84cfbe5f" containerName="pull"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384223    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebd79e96-7737-471d-8cc7-02ad84cfbe5f" containerName="pull"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384233    5626 cpu_manager.go:414] "containerMap: removing container" podUID="8b2301ed-5209-45f7-8262-51fb2f80ac33" containerName="util"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384238    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b2301ed-5209-45f7-8262-51fb2f80ac33" containerName="util"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384243    5626 cpu_manager.go:414] "containerMap: removing container" podUID="40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" containerName="extract"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384248    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" containerName="extract"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384256    5626 cpu_manager.go:414] "containerMap: removing container" podUID="9c102465-8050-4a38-8d87-a8149acf7f9f" containerName="util"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384260    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c102465-8050-4a38-8d87-a8149acf7f9f" containerName="util"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384267    5626 cpu_manager.go:414] "containerMap: removing container" podUID="9c102465-8050-4a38-8d87-a8149acf7f9f" containerName="pull"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384271    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c102465-8050-4a38-8d87-a8149acf7f9f" containerName="pull"
Nov 04 07:19:58.384261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384276    5626 cpu_manager.go:414] "containerMap: removing container" podUID="8b2301ed-5209-45f7-8262-51fb2f80ac33" containerName="extract"
Nov 04 07:19:58.385513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384281    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b2301ed-5209-45f7-8262-51fb2f80ac33" containerName="extract"
Nov 04 07:19:58.385513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384288    5626 cpu_manager.go:414] "containerMap: removing container" podUID="c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" containerName="extract"
Nov 04 07:19:58.385513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384294    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" containerName="extract"
Nov 04 07:19:58.385513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384302    5626 cpu_manager.go:414] "containerMap: removing container" podUID="ebd79e96-7737-471d-8cc7-02ad84cfbe5f" containerName="extract"
Nov 04 07:19:58.385513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384309    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebd79e96-7737-471d-8cc7-02ad84cfbe5f" containerName="extract"
Nov 04 07:19:58.385513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384390    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b2301ed-5209-45f7-8262-51fb2f80ac33" containerName="extract"
Nov 04 07:19:58.385513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384404    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3dab776-b13b-4e40-bcb3-f5d20d4d49a9" containerName="extract"
Nov 04 07:19:58.385513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384412    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebd79e96-7737-471d-8cc7-02ad84cfbe5f" containerName="extract"
Nov 04 07:19:58.385513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384423    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9c9d249-e95c-493c-aca6-710cec5bf1a5" containerName="extract"
Nov 04 07:19:58.385513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384431    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c102465-8050-4a38-8d87-a8149acf7f9f" containerName="extract"
Nov 04 07:19:58.385513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.384445    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="40c2f2af-1ecd-41f8-b849-2b44d37a5d9c" containerName="extract"
Nov 04 07:19:58.387918 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.387903    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.409659 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.409639    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 07:19:58.409820 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.409807    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 07:19:58.409902 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.409822    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 07:19:58.409902 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.409825    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-nhb6m\""
Nov 04 07:19:58.409902 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.409890    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 07:19:58.410026 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.409907    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 07:19:58.424750 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.424727    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"]
Nov 04 07:19:58.472322 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.472298    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq5g6\" (UniqueName: \"kubernetes.io/projected/b5d15c34-320f-4e4f-b159-fda262255cb7-kube-api-access-mq5g6\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.472449 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.472353    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5d15c34-320f-4e4f-b159-fda262255cb7-webhook-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.472490 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.472452    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5d15c34-320f-4e4f-b159-fda262255cb7-manifests\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.472535 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.472489    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5d15c34-320f-4e4f-b159-fda262255cb7-metrics-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.472535 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.472509    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5d15c34-320f-4e4f-b159-fda262255cb7-tmp\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.472615 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.472575    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5d15c34-320f-4e4f-b159-fda262255cb7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.573067 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.573037    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5d15c34-320f-4e4f-b159-fda262255cb7-tmp\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.573067 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.573069    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5d15c34-320f-4e4f-b159-fda262255cb7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.573302 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.573100    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq5g6\" (UniqueName: \"kubernetes.io/projected/b5d15c34-320f-4e4f-b159-fda262255cb7-kube-api-access-mq5g6\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.573302 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.573147    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5d15c34-320f-4e4f-b159-fda262255cb7-webhook-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.573302 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.573165    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5d15c34-320f-4e4f-b159-fda262255cb7-manifests\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.573302 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.573198    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5d15c34-320f-4e4f-b159-fda262255cb7-metrics-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.573594 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.573570    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5d15c34-320f-4e4f-b159-fda262255cb7-manifests\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.575348 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.575316    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5d15c34-320f-4e4f-b159-fda262255cb7-tmp\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.575647 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.575626    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5d15c34-320f-4e4f-b159-fda262255cb7-metrics-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.575784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.575764    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5d15c34-320f-4e4f-b159-fda262255cb7-webhook-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.575826 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.575773    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5d15c34-320f-4e4f-b159-fda262255cb7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.585722 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.585701    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq5g6\" (UniqueName: \"kubernetes.io/projected/b5d15c34-320f-4e4f-b159-fda262255cb7-kube-api-access-mq5g6\") pod \"opendatahub-operator-controller-manager-8f94f88d4-5nk76\" (UID: \"b5d15c34-320f-4e4f-b159-fda262255cb7\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:58.702418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:58.702344    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:19:59.001809 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:59.001613    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"]
Nov 04 07:19:59.001929 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:19:59.001903    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d15c34_320f_4e4f_b159_fda262255cb7.slice/crio-3bf3786cbf9a0a59837a5e9a8f4110297f456e6c4fe9187c86f235e1accaec6f WatchSource:0}: Error finding container 3bf3786cbf9a0a59837a5e9a8f4110297f456e6c4fe9187c86f235e1accaec6f: Status 404 returned error can't find the container with id 3bf3786cbf9a0a59837a5e9a8f4110297f456e6c4fe9187c86f235e1accaec6f
Nov 04 07:19:59.130663 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:19:59.130630    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76" event={"ID":"b5d15c34-320f-4e4f-b159-fda262255cb7","Type":"ContainerStarted","Data":"3bf3786cbf9a0a59837a5e9a8f4110297f456e6c4fe9187c86f235e1accaec6f"}
Nov 04 07:20:02.147454 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:02.147414    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76" event={"ID":"b5d15c34-320f-4e4f-b159-fda262255cb7","Type":"ContainerStarted","Data":"cc27b9c8284ca8abe42dcf0278de7a68101abacde2e0986c029f8fdb395b30e8"}
Nov 04 07:20:03.152988 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:03.152950    5626 generic.go:356] "Generic (PLEG): container finished" podID="b5d15c34-320f-4e4f-b159-fda262255cb7" containerID="cc27b9c8284ca8abe42dcf0278de7a68101abacde2e0986c029f8fdb395b30e8" exitCode=0
Nov 04 07:20:03.153521 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:03.153009    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76" event={"ID":"b5d15c34-320f-4e4f-b159-fda262255cb7","Type":"ContainerDied","Data":"cc27b9c8284ca8abe42dcf0278de7a68101abacde2e0986c029f8fdb395b30e8"}
Nov 04 07:20:04.159756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:04.159716    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76" event={"ID":"b5d15c34-320f-4e4f-b159-fda262255cb7","Type":"ContainerStarted","Data":"1278f393480bd7edd7aa79a1b1236a2b9028cee3df1759a2c506c257a01b23fa"}
Nov 04 07:20:04.160153 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:04.159828    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:20:04.216898 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:04.216846    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76" podStartSLOduration=2.316476344 podStartE2EDuration="6.216834392s" totalImagesPullingTime="3.410961286s" totalInitContainerRuntime="489.396762ms" isStatefulPod=false podCreationTimestamp="2026-11-04 07:19:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:20:04.215548657 +0000 UTC m=+226.237763908" watchObservedRunningTime="2026-11-04 07:20:04.216834392 +0000 UTC m=+226.239049651"
Nov 04 07:20:06.175970 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:20:06.175929    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:20:10.190872 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.190838    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"]
Nov 04 07:20:10.196978 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.196962    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"
Nov 04 07:20:10.205397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.205356    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 04 07:20:10.205530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.205516    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 04 07:20:10.205763 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.205741    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"]
Nov 04 07:20:10.272407 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.272355    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07041b7d-8ce2-4dfc-933e-e4bb7866b9c4-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-q9whl\" (UID: \"07041b7d-8ce2-4dfc-933e-e4bb7866b9c4\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"
Nov 04 07:20:10.272552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.272485    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gtkd\" (UniqueName: \"kubernetes.io/projected/07041b7d-8ce2-4dfc-933e-e4bb7866b9c4-kube-api-access-4gtkd\") pod \"cert-manager-webhook-5dbbbf75cc-q9whl\" (UID: \"07041b7d-8ce2-4dfc-933e-e4bb7866b9c4\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"
Nov 04 07:20:10.373440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.373409    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07041b7d-8ce2-4dfc-933e-e4bb7866b9c4-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-q9whl\" (UID: \"07041b7d-8ce2-4dfc-933e-e4bb7866b9c4\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"
Nov 04 07:20:10.373562 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.373486    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gtkd\" (UniqueName: \"kubernetes.io/projected/07041b7d-8ce2-4dfc-933e-e4bb7866b9c4-kube-api-access-4gtkd\") pod \"cert-manager-webhook-5dbbbf75cc-q9whl\" (UID: \"07041b7d-8ce2-4dfc-933e-e4bb7866b9c4\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"
Nov 04 07:20:10.385908 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.385879    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07041b7d-8ce2-4dfc-933e-e4bb7866b9c4-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-q9whl\" (UID: \"07041b7d-8ce2-4dfc-933e-e4bb7866b9c4\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"
Nov 04 07:20:10.386205 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.386180    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gtkd\" (UniqueName: \"kubernetes.io/projected/07041b7d-8ce2-4dfc-933e-e4bb7866b9c4-kube-api-access-4gtkd\") pod \"cert-manager-webhook-5dbbbf75cc-q9whl\" (UID: \"07041b7d-8ce2-4dfc-933e-e4bb7866b9c4\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"
Nov 04 07:20:10.510161 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.510088    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"
Nov 04 07:20:10.790749 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:10.790720    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"]
Nov 04 07:20:10.792694 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:20:10.792669    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07041b7d_8ce2_4dfc_933e_e4bb7866b9c4.slice/crio-8266a656421f4eb54c8ad0e7be6aaa91be9756263e386cd761e4444ebf78734f WatchSource:0}: Error finding container 8266a656421f4eb54c8ad0e7be6aaa91be9756263e386cd761e4444ebf78734f: Status 404 returned error can't find the container with id 8266a656421f4eb54c8ad0e7be6aaa91be9756263e386cd761e4444ebf78734f
Nov 04 07:20:11.191763 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:11.191674    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl" event={"ID":"07041b7d-8ce2-4dfc-933e-e4bb7866b9c4","Type":"ContainerStarted","Data":"8266a656421f4eb54c8ad0e7be6aaa91be9756263e386cd761e4444ebf78734f"}
Nov 04 07:20:12.496950 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:12.496916    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55"]
Nov 04 07:20:12.500491 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:12.500471    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55"
Nov 04 07:20:12.511377 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:12.511329    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-bqxcb\""
Nov 04 07:20:12.511722 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:12.511703    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55"]
Nov 04 07:20:12.591499 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:12.591459    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3939b90-6cf5-4733-a086-71a0393f707f-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-sqp55\" (UID: \"d3939b90-6cf5-4733-a086-71a0393f707f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55"
Nov 04 07:20:12.591697 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:12.591513    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t66m\" (UniqueName: \"kubernetes.io/projected/d3939b90-6cf5-4733-a086-71a0393f707f-kube-api-access-4t66m\") pod \"cert-manager-cainjector-5c57bbd4fc-sqp55\" (UID: \"d3939b90-6cf5-4733-a086-71a0393f707f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55"
Nov 04 07:20:12.692550 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:12.692512    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3939b90-6cf5-4733-a086-71a0393f707f-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-sqp55\" (UID: \"d3939b90-6cf5-4733-a086-71a0393f707f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55"
Nov 04 07:20:12.692743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:12.692561    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t66m\" (UniqueName: \"kubernetes.io/projected/d3939b90-6cf5-4733-a086-71a0393f707f-kube-api-access-4t66m\") pod \"cert-manager-cainjector-5c57bbd4fc-sqp55\" (UID: \"d3939b90-6cf5-4733-a086-71a0393f707f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55"
Nov 04 07:20:12.706876 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:12.706841    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t66m\" (UniqueName: \"kubernetes.io/projected/d3939b90-6cf5-4733-a086-71a0393f707f-kube-api-access-4t66m\") pod \"cert-manager-cainjector-5c57bbd4fc-sqp55\" (UID: \"d3939b90-6cf5-4733-a086-71a0393f707f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55"
Nov 04 07:20:12.707168 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:12.707145    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3939b90-6cf5-4733-a086-71a0393f707f-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-sqp55\" (UID: \"d3939b90-6cf5-4733-a086-71a0393f707f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55"
Nov 04 07:20:12.814216 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:12.814179    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55"
Nov 04 07:20:13.106138 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:13.105981    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55"]
Nov 04 07:20:13.109274 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:20:13.109236    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3939b90_6cf5_4733_a086_71a0393f707f.slice/crio-653b992ca3f4f11a4ac8cdb6d59d7a2fd26cf9a93acfe82aa5ae5eb38ecc4ea9 WatchSource:0}: Error finding container 653b992ca3f4f11a4ac8cdb6d59d7a2fd26cf9a93acfe82aa5ae5eb38ecc4ea9: Status 404 returned error can't find the container with id 653b992ca3f4f11a4ac8cdb6d59d7a2fd26cf9a93acfe82aa5ae5eb38ecc4ea9
Nov 04 07:20:13.201769 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:13.201732    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55" event={"ID":"d3939b90-6cf5-4733-a086-71a0393f707f","Type":"ContainerStarted","Data":"653b992ca3f4f11a4ac8cdb6d59d7a2fd26cf9a93acfe82aa5ae5eb38ecc4ea9"}
Nov 04 07:20:14.208392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:14.208286    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl" event={"ID":"07041b7d-8ce2-4dfc-933e-e4bb7866b9c4","Type":"ContainerStarted","Data":"f20ddabbd415ab83afbb33a4e44d6db3cbcea3210805c6f3d64d36c6a9d1d77b"}
Nov 04 07:20:14.208392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:14.208383    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"
Nov 04 07:20:14.253562 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:14.253508    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl" podStartSLOduration=1.098516995 podStartE2EDuration="4.25349109s" totalImagesPullingTime="3.154974095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:20:14.251552129 +0000 UTC m=+236.273767389" watchObservedRunningTime="2026-11-04 07:20:14.25349109 +0000 UTC m=+236.275706351"
Nov 04 07:20:15.166574 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:15.166485    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76"
Nov 04 07:20:15.215206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:15.215143    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55" event={"ID":"d3939b90-6cf5-4733-a086-71a0393f707f","Type":"ContainerStarted","Data":"7c34b0121fe6899949da9d6ee1c99b37e90391d3444b3745d079f024e9ad11aa"}
Nov 04 07:20:20.217822 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:20.217793    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl"
Nov 04 07:20:20.240047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:20.240002    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55" podStartSLOduration=6.493803108 podStartE2EDuration="8.23998934s" totalImagesPullingTime="1.746186232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:20:15.242469836 +0000 UTC m=+237.264685109" watchObservedRunningTime="2026-11-04 07:20:20.23998934 +0000 UTC m=+242.262204594"
Nov 04 07:20:21.250847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:21.250812    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-qpmxd"]
Nov 04 07:20:21.255924 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:21.255899    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-qpmxd"
Nov 04 07:20:21.283511 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:21.283487    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-6wrwz\""
Nov 04 07:20:21.290432 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:21.290405    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-qpmxd"]
Nov 04 07:20:21.371420 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:21.371373    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlmc9\" (UniqueName: \"kubernetes.io/projected/13ec8661-cef2-4782-a530-d781db896e56-kube-api-access-wlmc9\") pod \"cert-manager-597b95cb84-qpmxd\" (UID: \"13ec8661-cef2-4782-a530-d781db896e56\") " pod="cert-manager/cert-manager-597b95cb84-qpmxd"
Nov 04 07:20:21.371646 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:21.371624    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13ec8661-cef2-4782-a530-d781db896e56-bound-sa-token\") pod \"cert-manager-597b95cb84-qpmxd\" (UID: \"13ec8661-cef2-4782-a530-d781db896e56\") " pod="cert-manager/cert-manager-597b95cb84-qpmxd"
Nov 04 07:20:21.472620 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:21.472584    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13ec8661-cef2-4782-a530-d781db896e56-bound-sa-token\") pod \"cert-manager-597b95cb84-qpmxd\" (UID: \"13ec8661-cef2-4782-a530-d781db896e56\") " pod="cert-manager/cert-manager-597b95cb84-qpmxd"
Nov 04 07:20:21.473273 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:21.473249    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlmc9\" (UniqueName: \"kubernetes.io/projected/13ec8661-cef2-4782-a530-d781db896e56-kube-api-access-wlmc9\") pod \"cert-manager-597b95cb84-qpmxd\" (UID: \"13ec8661-cef2-4782-a530-d781db896e56\") " pod="cert-manager/cert-manager-597b95cb84-qpmxd"
Nov 04 07:20:21.487253 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:21.487232    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13ec8661-cef2-4782-a530-d781db896e56-bound-sa-token\") pod \"cert-manager-597b95cb84-qpmxd\" (UID: \"13ec8661-cef2-4782-a530-d781db896e56\") " pod="cert-manager/cert-manager-597b95cb84-qpmxd"
Nov 04 07:20:21.487384 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:21.487293    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlmc9\" (UniqueName: \"kubernetes.io/projected/13ec8661-cef2-4782-a530-d781db896e56-kube-api-access-wlmc9\") pod \"cert-manager-597b95cb84-qpmxd\" (UID: \"13ec8661-cef2-4782-a530-d781db896e56\") " pod="cert-manager/cert-manager-597b95cb84-qpmxd"
Nov 04 07:20:21.565597 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:21.565575    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-qpmxd"
Nov 04 07:20:21.838173 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:21.838098    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-qpmxd"]
Nov 04 07:20:21.841307 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:20:21.841278    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13ec8661_cef2_4782_a530_d781db896e56.slice/crio-cf408c8de9c9676b88b4b9def102952d4b0227ea83e63c013a0c926001e00bb6 WatchSource:0}: Error finding container cf408c8de9c9676b88b4b9def102952d4b0227ea83e63c013a0c926001e00bb6: Status 404 returned error can't find the container with id cf408c8de9c9676b88b4b9def102952d4b0227ea83e63c013a0c926001e00bb6
Nov 04 07:20:22.246165 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:22.246083    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-qpmxd" event={"ID":"13ec8661-cef2-4782-a530-d781db896e56","Type":"ContainerStarted","Data":"7baa666598a458d138ddad6e2981f63e8381344d7f50f74d3f5bb1e606848fe8"}
Nov 04 07:20:22.246165 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:22.246123    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-qpmxd" event={"ID":"13ec8661-cef2-4782-a530-d781db896e56","Type":"ContainerStarted","Data":"cf408c8de9c9676b88b4b9def102952d4b0227ea83e63c013a0c926001e00bb6"}
Nov 04 07:20:22.306800 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:22.306744    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-qpmxd" podStartSLOduration=1.306727666 podStartE2EDuration="1.306727666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:20:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:20:22.305915466 +0000 UTC m=+244.328130725" watchObservedRunningTime="2026-11-04 07:20:22.306727666 +0000 UTC m=+244.328942926"
Nov 04 07:20:24.117064 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:24.117026    5626 pod_container_manager_linux.go:218] "Failed to delete cgroup paths" cgroupName=["kubepods","burstable","podebd79e96-7737-471d-8cc7-02ad84cfbe5f"] err="unable to destroy cgroup paths for cgroup [kubepods burstable podebd79e96-7737-471d-8cc7-02ad84cfbe5f] : Timed out while waiting for systemd to remove kubepods-burstable-podebd79e96_7737_471d_8cc7_02ad84cfbe5f.slice"
Nov 04 07:20:24.117064 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:24.117059    5626 kubelet_pods.go:2797] "Failed to destroy orphaned pod cgroup" podUID="ebd79e96-7737-471d-8cc7-02ad84cfbe5f" err="failed to delete cgroup paths for [kubepods burstable podebd79e96-7737-471d-8cc7-02ad84cfbe5f] : unable to destroy cgroup paths for cgroup [kubepods burstable podebd79e96-7737-471d-8cc7-02ad84cfbe5f] : Timed out while waiting for systemd to remove kubepods-burstable-podebd79e96_7737_471d_8cc7_02ad84cfbe5f.slice"
Nov 04 07:20:31.186408 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.186355    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"]
Nov 04 07:20:31.189583 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.189563    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"
Nov 04 07:20:31.216480 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.216455    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 04 07:20:31.216721 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.216700    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mvxk2\""
Nov 04 07:20:31.217134 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.217109    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 04 07:20:31.220303 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.220278    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"]
Nov 04 07:20:31.357289 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.357254    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78a42108-2b93-415c-a7e2-075603f71c38-tmp\") pod \"kube-auth-proxy-589fcc7ff4-h6ggf\" (UID: \"78a42108-2b93-415c-a7e2-075603f71c38\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"
Nov 04 07:20:31.357289 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.357291    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78a42108-2b93-415c-a7e2-075603f71c38-tls-certs\") pod \"kube-auth-proxy-589fcc7ff4-h6ggf\" (UID: \"78a42108-2b93-415c-a7e2-075603f71c38\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"
Nov 04 07:20:31.357652 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.357332    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95hfk\" (UniqueName: \"kubernetes.io/projected/78a42108-2b93-415c-a7e2-075603f71c38-kube-api-access-95hfk\") pod \"kube-auth-proxy-589fcc7ff4-h6ggf\" (UID: \"78a42108-2b93-415c-a7e2-075603f71c38\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"
Nov 04 07:20:31.458157 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.458069    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78a42108-2b93-415c-a7e2-075603f71c38-tmp\") pod \"kube-auth-proxy-589fcc7ff4-h6ggf\" (UID: \"78a42108-2b93-415c-a7e2-075603f71c38\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"
Nov 04 07:20:31.458157 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.458107    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78a42108-2b93-415c-a7e2-075603f71c38-tls-certs\") pod \"kube-auth-proxy-589fcc7ff4-h6ggf\" (UID: \"78a42108-2b93-415c-a7e2-075603f71c38\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"
Nov 04 07:20:31.458157 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.458154    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95hfk\" (UniqueName: \"kubernetes.io/projected/78a42108-2b93-415c-a7e2-075603f71c38-kube-api-access-95hfk\") pod \"kube-auth-proxy-589fcc7ff4-h6ggf\" (UID: \"78a42108-2b93-415c-a7e2-075603f71c38\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"
Nov 04 07:20:31.460463 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.460437    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78a42108-2b93-415c-a7e2-075603f71c38-tmp\") pod \"kube-auth-proxy-589fcc7ff4-h6ggf\" (UID: \"78a42108-2b93-415c-a7e2-075603f71c38\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"
Nov 04 07:20:31.460596 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.460579    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78a42108-2b93-415c-a7e2-075603f71c38-tls-certs\") pod \"kube-auth-proxy-589fcc7ff4-h6ggf\" (UID: \"78a42108-2b93-415c-a7e2-075603f71c38\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"
Nov 04 07:20:31.474081 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.474061    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95hfk\" (UniqueName: \"kubernetes.io/projected/78a42108-2b93-415c-a7e2-075603f71c38-kube-api-access-95hfk\") pod \"kube-auth-proxy-589fcc7ff4-h6ggf\" (UID: \"78a42108-2b93-415c-a7e2-075603f71c38\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"
Nov 04 07:20:31.501026 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.501001    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"
Nov 04 07:20:31.775845 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:31.775815    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf"]
Nov 04 07:20:31.778205 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:20:31.778175    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a42108_2b93_415c_a7e2_075603f71c38.slice/crio-230d8f6eae3cacf59781298d2ad69832fe3ef21038436d13977bd1f0e01b6261 WatchSource:0}: Error finding container 230d8f6eae3cacf59781298d2ad69832fe3ef21038436d13977bd1f0e01b6261: Status 404 returned error can't find the container with id 230d8f6eae3cacf59781298d2ad69832fe3ef21038436d13977bd1f0e01b6261
Nov 04 07:20:32.288204 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:32.288169    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf" event={"ID":"78a42108-2b93-415c-a7e2-075603f71c38","Type":"ContainerStarted","Data":"230d8f6eae3cacf59781298d2ad69832fe3ef21038436d13977bd1f0e01b6261"}
Nov 04 07:20:35.304575 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:35.304539    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf" event={"ID":"78a42108-2b93-415c-a7e2-075603f71c38","Type":"ContainerStarted","Data":"de5e965cb7ada0ba9b9a07406851011c263d0bb8f084c96bc39916bde7a5222d"}
Nov 04 07:20:35.325791 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:20:35.325743    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf" podStartSLOduration=1.786693933 podStartE2EDuration="4.325731001s" totalImagesPullingTime="2.539037068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:20:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:20:35.324459473 +0000 UTC m=+257.346674734" watchObservedRunningTime="2026-11-04 07:20:35.325731001 +0000 UTC m=+257.347946260"
Nov 04 07:20:36.170942 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:20:36.170899    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:21:06.173865 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:21:06.173771    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:21:27.068466 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.068431    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-msppb"]
Nov 04 07:21:27.075936 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.075905    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-msppb"
Nov 04 07:21:27.084964 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.084932    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-msppb"]
Nov 04 07:21:27.086956 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.086937    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Nov 04 07:21:27.087685 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.087664    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-6lglc\""
Nov 04 07:21:27.140754 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.140723    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 07:21:27.140904 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.140818    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 07:21:27.223515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.223482    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66kjc\" (UniqueName: \"kubernetes.io/projected/2d27b8cd-6f7a-4657-9105-ccc5d7b05652-kube-api-access-66kjc\") pod \"model-serving-api-75bd89fdfc-msppb\" (UID: \"2d27b8cd-6f7a-4657-9105-ccc5d7b05652\") " pod="opendatahub/model-serving-api-75bd89fdfc-msppb"
Nov 04 07:21:27.223682 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.223618    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d27b8cd-6f7a-4657-9105-ccc5d7b05652-tls-certs\") pod \"model-serving-api-75bd89fdfc-msppb\" (UID: \"2d27b8cd-6f7a-4657-9105-ccc5d7b05652\") " pod="opendatahub/model-serving-api-75bd89fdfc-msppb"
Nov 04 07:21:27.324584 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.324499    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66kjc\" (UniqueName: \"kubernetes.io/projected/2d27b8cd-6f7a-4657-9105-ccc5d7b05652-kube-api-access-66kjc\") pod \"model-serving-api-75bd89fdfc-msppb\" (UID: \"2d27b8cd-6f7a-4657-9105-ccc5d7b05652\") " pod="opendatahub/model-serving-api-75bd89fdfc-msppb"
Nov 04 07:21:27.324584 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.324556    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d27b8cd-6f7a-4657-9105-ccc5d7b05652-tls-certs\") pod \"model-serving-api-75bd89fdfc-msppb\" (UID: \"2d27b8cd-6f7a-4657-9105-ccc5d7b05652\") " pod="opendatahub/model-serving-api-75bd89fdfc-msppb"
Nov 04 07:21:27.330933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.330901    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d27b8cd-6f7a-4657-9105-ccc5d7b05652-tls-certs\") pod \"model-serving-api-75bd89fdfc-msppb\" (UID: \"2d27b8cd-6f7a-4657-9105-ccc5d7b05652\") " pod="opendatahub/model-serving-api-75bd89fdfc-msppb"
Nov 04 07:21:27.349342 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.349315    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66kjc\" (UniqueName: \"kubernetes.io/projected/2d27b8cd-6f7a-4657-9105-ccc5d7b05652-kube-api-access-66kjc\") pod \"model-serving-api-75bd89fdfc-msppb\" (UID: \"2d27b8cd-6f7a-4657-9105-ccc5d7b05652\") " pod="opendatahub/model-serving-api-75bd89fdfc-msppb"
Nov 04 07:21:27.397375 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.397328    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-msppb"
Nov 04 07:21:27.683972 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:27.683908    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-msppb"]
Nov 04 07:21:27.686413 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:21:27.686383    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d27b8cd_6f7a_4657_9105_ccc5d7b05652.slice/crio-a7eb6980350b6f6f3de66fec63be3a04fc10a738141641bdc7db420fa646af83 WatchSource:0}: Error finding container a7eb6980350b6f6f3de66fec63be3a04fc10a738141641bdc7db420fa646af83: Status 404 returned error can't find the container with id a7eb6980350b6f6f3de66fec63be3a04fc10a738141641bdc7db420fa646af83
Nov 04 07:21:28.561535 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:28.561498    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-msppb" event={"ID":"2d27b8cd-6f7a-4657-9105-ccc5d7b05652","Type":"ContainerStarted","Data":"a7eb6980350b6f6f3de66fec63be3a04fc10a738141641bdc7db420fa646af83"}
Nov 04 07:21:29.568133 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:29.568095    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-msppb" event={"ID":"2d27b8cd-6f7a-4657-9105-ccc5d7b05652","Type":"ContainerStarted","Data":"e138b3ae234982df326c05727e148fe09bbb4cf50b71a5f242cacb157dae055d"}
Nov 04 07:21:29.568564 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:29.568203    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-msppb"
Nov 04 07:21:29.601430 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:29.601321    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-msppb" podStartSLOduration=0.966947952 podStartE2EDuration="2.601307884s" totalImagesPullingTime="1.634359932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:21:29.600950746 +0000 UTC m=+311.623166007" watchObservedRunningTime="2026-11-04 07:21:29.601307884 +0000 UTC m=+311.623523144"
Nov 04 07:21:35.392823 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.392791    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"]
Nov 04 07:21:35.396128 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.396110    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.402308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.402277    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Nov 04 07:21:35.402476 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.402339    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Nov 04 07:21:35.402476 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.402424    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-sxsbr\""
Nov 04 07:21:35.402891 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.402519    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Nov 04 07:21:35.406627 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.406602    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"]
Nov 04 07:21:35.501181 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.501149    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4fb3aedb-6fac-44fa-8823-d926fa0d71f9-metrics-certs\") pod \"jobset-controller-manager-6d445bbc67-m4gfl\" (UID: \"4fb3aedb-6fac-44fa-8823-d926fa0d71f9\") " pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.501336 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.501195    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4fb3aedb-6fac-44fa-8823-d926fa0d71f9-manager-config\") pod \"jobset-controller-manager-6d445bbc67-m4gfl\" (UID: \"4fb3aedb-6fac-44fa-8823-d926fa0d71f9\") " pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.501336 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.501220    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntttj\" (UniqueName: \"kubernetes.io/projected/4fb3aedb-6fac-44fa-8823-d926fa0d71f9-kube-api-access-ntttj\") pod \"jobset-controller-manager-6d445bbc67-m4gfl\" (UID: \"4fb3aedb-6fac-44fa-8823-d926fa0d71f9\") " pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.501450 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.501386    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fb3aedb-6fac-44fa-8823-d926fa0d71f9-cert\") pod \"jobset-controller-manager-6d445bbc67-m4gfl\" (UID: \"4fb3aedb-6fac-44fa-8823-d926fa0d71f9\") " pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.602722 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.602692    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4fb3aedb-6fac-44fa-8823-d926fa0d71f9-metrics-certs\") pod \"jobset-controller-manager-6d445bbc67-m4gfl\" (UID: \"4fb3aedb-6fac-44fa-8823-d926fa0d71f9\") " pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.602902 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.602739    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4fb3aedb-6fac-44fa-8823-d926fa0d71f9-manager-config\") pod \"jobset-controller-manager-6d445bbc67-m4gfl\" (UID: \"4fb3aedb-6fac-44fa-8823-d926fa0d71f9\") " pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.602902 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.602762    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntttj\" (UniqueName: \"kubernetes.io/projected/4fb3aedb-6fac-44fa-8823-d926fa0d71f9-kube-api-access-ntttj\") pod \"jobset-controller-manager-6d445bbc67-m4gfl\" (UID: \"4fb3aedb-6fac-44fa-8823-d926fa0d71f9\") " pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.602902 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.602814    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fb3aedb-6fac-44fa-8823-d926fa0d71f9-cert\") pod \"jobset-controller-manager-6d445bbc67-m4gfl\" (UID: \"4fb3aedb-6fac-44fa-8823-d926fa0d71f9\") " pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.603374 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.603344    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4fb3aedb-6fac-44fa-8823-d926fa0d71f9-manager-config\") pod \"jobset-controller-manager-6d445bbc67-m4gfl\" (UID: \"4fb3aedb-6fac-44fa-8823-d926fa0d71f9\") " pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.605256 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.605227    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4fb3aedb-6fac-44fa-8823-d926fa0d71f9-metrics-certs\") pod \"jobset-controller-manager-6d445bbc67-m4gfl\" (UID: \"4fb3aedb-6fac-44fa-8823-d926fa0d71f9\") " pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.605352 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.605285    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fb3aedb-6fac-44fa-8823-d926fa0d71f9-cert\") pod \"jobset-controller-manager-6d445bbc67-m4gfl\" (UID: \"4fb3aedb-6fac-44fa-8823-d926fa0d71f9\") " pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.615566 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.615544    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntttj\" (UniqueName: \"kubernetes.io/projected/4fb3aedb-6fac-44fa-8823-d926fa0d71f9-kube-api-access-ntttj\") pod \"jobset-controller-manager-6d445bbc67-m4gfl\" (UID: \"4fb3aedb-6fac-44fa-8823-d926fa0d71f9\") " pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.708139 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.708057    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:35.982483 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:35.982461    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"]
Nov 04 07:21:35.984887 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:21:35.984858    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb3aedb_6fac_44fa_8823_d926fa0d71f9.slice/crio-0940d0e86999b39713035121c0e34bfea84259cbf607e5bac7760048ebe1c08e WatchSource:0}: Error finding container 0940d0e86999b39713035121c0e34bfea84259cbf607e5bac7760048ebe1c08e: Status 404 returned error can't find the container with id 0940d0e86999b39713035121c0e34bfea84259cbf607e5bac7760048ebe1c08e
Nov 04 07:21:36.170997 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:21:36.170953    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:21:36.604804 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:36.604772    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl" event={"ID":"4fb3aedb-6fac-44fa-8823-d926fa0d71f9","Type":"ContainerStarted","Data":"0940d0e86999b39713035121c0e34bfea84259cbf607e5bac7760048ebe1c08e"}
Nov 04 07:21:40.577796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:40.577766    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-msppb"
Nov 04 07:21:40.626653 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:40.626621    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl" event={"ID":"4fb3aedb-6fac-44fa-8823-d926fa0d71f9","Type":"ContainerStarted","Data":"cc7f6f5ca6530990206774404a7f983b5f4f3a5a0716cb750948ceebb02ee034"}
Nov 04 07:21:40.626845 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:40.626736    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:21:40.646854 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:40.646805    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl" podStartSLOduration=1.926480486 podStartE2EDuration="5.646789379s" totalImagesPullingTime="3.720308893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:21:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:21:40.645521818 +0000 UTC m=+322.667737078" watchObservedRunningTime="2026-11-04 07:21:40.646789379 +0000 UTC m=+322.669004638"
Nov 04 07:21:51.636024 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:21:51.635993    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl"
Nov 04 07:22:06.171392 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:22:06.171339    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:22:36.171237 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:22:36.171156    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:23:06.171536 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:23:06.171497    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:23:36.171765 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:23:36.171724    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:24:06.170227 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:24:06.170138    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:24:36.172452 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:24:36.172414    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:25:06.171215 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:25:06.171170    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:25:36.171860 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:25:36.171754    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:26:06.171035 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:26:06.170994    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:26:36.171387 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:26:36.171321    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:27:06.171621 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:27:06.171543    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:27:36.171675 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:27:36.171638    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:28:06.172072 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:28:06.172030    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:28:36.170848 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:28:36.170768    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:29:06.171678 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:29:06.171637    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:29:36.171495 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:29:36.171456    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:30:06.172461 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:30:06.172377    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:30:36.172013 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:30:36.171899    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:31:06.172297 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:31:06.172251    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:31:10.245984 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:10.245950    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-llg5c"]
Nov 04 07:31:10.249212 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:10.249195    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-llg5c"
Nov 04 07:31:10.257426 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:10.257401    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-llg5c"]
Nov 04 07:31:10.257548 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:10.257529    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 07:31:10.257628 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:10.257601    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 07:31:10.257670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:10.257633    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c9nbb\""
Nov 04 07:31:10.318574 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:10.318548    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mddn8\" (UniqueName: \"kubernetes.io/projected/d835348b-dfc8-4020-8bf1-5fe2b06c97f1-kube-api-access-mddn8\") pod \"authorino-56c6c9ddd5-llg5c\" (UID: \"d835348b-dfc8-4020-8bf1-5fe2b06c97f1\") " pod="kuadrant-system/authorino-56c6c9ddd5-llg5c"
Nov 04 07:31:10.419728 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:10.419688    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mddn8\" (UniqueName: \"kubernetes.io/projected/d835348b-dfc8-4020-8bf1-5fe2b06c97f1-kube-api-access-mddn8\") pod \"authorino-56c6c9ddd5-llg5c\" (UID: \"d835348b-dfc8-4020-8bf1-5fe2b06c97f1\") " pod="kuadrant-system/authorino-56c6c9ddd5-llg5c"
Nov 04 07:31:10.432712 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:10.432691    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mddn8\" (UniqueName: \"kubernetes.io/projected/d835348b-dfc8-4020-8bf1-5fe2b06c97f1-kube-api-access-mddn8\") pod \"authorino-56c6c9ddd5-llg5c\" (UID: \"d835348b-dfc8-4020-8bf1-5fe2b06c97f1\") " pod="kuadrant-system/authorino-56c6c9ddd5-llg5c"
Nov 04 07:31:10.561485 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:10.561458    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-llg5c"
Nov 04 07:31:10.838477 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:10.838451    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-llg5c"]
Nov 04 07:31:10.841376 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:31:10.841327    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd835348b_dfc8_4020_8bf1_5fe2b06c97f1.slice/crio-adc93a730406648c992b37188926e77fc4b5a2c20f8e29010ff464406a7ce431 WatchSource:0}: Error finding container adc93a730406648c992b37188926e77fc4b5a2c20f8e29010ff464406a7ce431: Status 404 returned error can't find the container with id adc93a730406648c992b37188926e77fc4b5a2c20f8e29010ff464406a7ce431
Nov 04 07:31:10.842655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:10.842632    5626 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 07:31:11.176187 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:11.176111    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-llg5c" event={"ID":"d835348b-dfc8-4020-8bf1-5fe2b06c97f1","Type":"ContainerStarted","Data":"adc93a730406648c992b37188926e77fc4b5a2c20f8e29010ff464406a7ce431"}
Nov 04 07:31:14.193283 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:14.193242    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-llg5c" event={"ID":"d835348b-dfc8-4020-8bf1-5fe2b06c97f1","Type":"ContainerStarted","Data":"d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719"}
Nov 04 07:31:14.210974 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:31:14.210925    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-llg5c" podStartSLOduration=1.483314072 podStartE2EDuration="4.210911313s" totalImagesPullingTime="2.727597241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:31:14.210029318 +0000 UTC m=+896.232244579" watchObservedRunningTime="2026-11-04 07:31:14.210911313 +0000 UTC m=+896.233126624"
Nov 04 07:31:36.172566 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:31:36.172480    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:32:06.172124 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:32:06.172088    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:32:36.174035 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:32:36.173930    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:32:38.915907 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:38.915860    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-xgjgh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:32:38.916346 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:38.915934    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh" podUID="7ba40135-db97-4238-8bb8-f1fd581a1615" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:32:39.611399 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:39.611342    5626 generic.go:356] "Generic (PLEG): container finished" podID="7ba40135-db97-4238-8bb8-f1fd581a1615" containerID="a0302d10b93d4ea4d71ac43b26329a2c6e6984c5fd0019630cd38d1a2bc3fcac" exitCode=0
Nov 04 07:32:39.611568 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:39.611418    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh" event={"ID":"7ba40135-db97-4238-8bb8-f1fd581a1615","Type":"ContainerDied","Data":"a0302d10b93d4ea4d71ac43b26329a2c6e6984c5fd0019630cd38d1a2bc3fcac"}
Nov 04 07:32:39.611821 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:39.611800    5626 scope.go:122] "RemoveContainer" containerID="a0302d10b93d4ea4d71ac43b26329a2c6e6984c5fd0019630cd38d1a2bc3fcac"
Nov 04 07:32:39.612869 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:39.612839    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:32:39.612968 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:39.612905    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:32:39.612968 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:39.612937    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:32:40.425058 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.425021    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-4m66s"]
Nov 04 07:32:40.428630 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.428608    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-4m66s"
Nov 04 07:32:40.453386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.439141    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-4m66s"]
Nov 04 07:32:40.483581 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.483554    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Nov 04 07:32:40.520250 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.520222    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkp8f\" (UniqueName: \"kubernetes.io/projected/80d949ec-572e-4e86-b860-b09b612085e8-kube-api-access-zkp8f\") pod \"authorino-6889cbd8c4-4m66s\" (UID: \"80d949ec-572e-4e86-b860-b09b612085e8\") " pod="kuadrant-system/authorino-6889cbd8c4-4m66s"
Nov 04 07:32:40.520458 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.520257    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80d949ec-572e-4e86-b860-b09b612085e8-tls-cert\") pod \"authorino-6889cbd8c4-4m66s\" (UID: \"80d949ec-572e-4e86-b860-b09b612085e8\") " pod="kuadrant-system/authorino-6889cbd8c4-4m66s"
Nov 04 07:32:40.617975 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.617940    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh" event={"ID":"7ba40135-db97-4238-8bb8-f1fd581a1615","Type":"ContainerStarted","Data":"56104f319df23d8de336046a9ab4b1fc990a453f0d0cc90a95d804bd92303106"}
Nov 04 07:32:40.621610 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.621581    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80d949ec-572e-4e86-b860-b09b612085e8-tls-cert\") pod \"authorino-6889cbd8c4-4m66s\" (UID: \"80d949ec-572e-4e86-b860-b09b612085e8\") " pod="kuadrant-system/authorino-6889cbd8c4-4m66s"
Nov 04 07:32:40.621735 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.621680    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkp8f\" (UniqueName: \"kubernetes.io/projected/80d949ec-572e-4e86-b860-b09b612085e8-kube-api-access-zkp8f\") pod \"authorino-6889cbd8c4-4m66s\" (UID: \"80d949ec-572e-4e86-b860-b09b612085e8\") " pod="kuadrant-system/authorino-6889cbd8c4-4m66s"
Nov 04 07:32:40.623986 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.623964    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80d949ec-572e-4e86-b860-b09b612085e8-tls-cert\") pod \"authorino-6889cbd8c4-4m66s\" (UID: \"80d949ec-572e-4e86-b860-b09b612085e8\") " pod="kuadrant-system/authorino-6889cbd8c4-4m66s"
Nov 04 07:32:40.635026 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.635003    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkp8f\" (UniqueName: \"kubernetes.io/projected/80d949ec-572e-4e86-b860-b09b612085e8-kube-api-access-zkp8f\") pod \"authorino-6889cbd8c4-4m66s\" (UID: \"80d949ec-572e-4e86-b860-b09b612085e8\") " pod="kuadrant-system/authorino-6889cbd8c4-4m66s"
Nov 04 07:32:40.711315 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.711232    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-4m66s"]
Nov 04 07:32:40.711487 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:40.711473    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-4m66s"
Nov 04 07:32:41.005772 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:41.005747    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-4m66s"]
Nov 04 07:32:41.008208 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:32:41.008178    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80d949ec_572e_4e86_b860_b09b612085e8.slice/crio-083be772122c6c2638ee3a66e2c6d8bec0f81d4e65857d4ca45a933861024661 WatchSource:0}: Error finding container 083be772122c6c2638ee3a66e2c6d8bec0f81d4e65857d4ca45a933861024661: Status 404 returned error can't find the container with id 083be772122c6c2638ee3a66e2c6d8bec0f81d4e65857d4ca45a933861024661
Nov 04 07:32:41.584547 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:41.584511    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:32:41.585021 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:41.584639    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-xgjgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Nov 04 07:32:41.585021 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:41.584714    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh" podUID="7ba40135-db97-4238-8bb8-f1fd581a1615" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Nov 04 07:32:41.623905 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:41.623873    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-4m66s" event={"ID":"80d949ec-572e-4e86-b860-b09b612085e8","Type":"ContainerStarted","Data":"083be772122c6c2638ee3a66e2c6d8bec0f81d4e65857d4ca45a933861024661"}
Nov 04 07:32:42.584572 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:42.584529    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-xgjgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Nov 04 07:32:42.585069 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:42.584583    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh" podUID="7ba40135-db97-4238-8bb8-f1fd581a1615" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Nov 04 07:32:42.632027 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:42.631994    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-4m66s" event={"ID":"80d949ec-572e-4e86-b860-b09b612085e8","Type":"ContainerStarted","Data":"206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c"}
Nov 04 07:32:42.632204 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:42.632032    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6889cbd8c4-4m66s" podUID="80d949ec-572e-4e86-b860-b09b612085e8" containerName="authorino" containerID="cri-o://206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c" gracePeriod=30
Nov 04 07:32:42.732009 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:42.731957    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6889cbd8c4-4m66s" podStartSLOduration=1.5736179 podStartE2EDuration="2.731940537s" totalImagesPullingTime="1.158322637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:32:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:32:42.657130638 +0000 UTC m=+984.679345901" watchObservedRunningTime="2026-11-04 07:32:42.731940537 +0000 UTC m=+984.754155797"
Nov 04 07:32:42.732989 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:42.732965    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-llg5c"]
Nov 04 07:32:42.733148 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:42.733127    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-llg5c" podUID="d835348b-dfc8-4020-8bf1-5fe2b06c97f1" containerName="authorino" containerID="cri-o://d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719" gracePeriod=30
Nov 04 07:32:42.905175 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:42.905143    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-4m66s"
Nov 04 07:32:43.001982 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.001957    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-llg5c"
Nov 04 07:32:43.070458 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.070427    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80d949ec-572e-4e86-b860-b09b612085e8-kube-api-access-zkp8f\" (UniqueName: \"kubernetes.io/projected/80d949ec-572e-4e86-b860-b09b612085e8-kube-api-access-zkp8f\") pod \"80d949ec-572e-4e86-b860-b09b612085e8\" (UID: \"80d949ec-572e-4e86-b860-b09b612085e8\") "
Nov 04 07:32:43.070656 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.070506    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80d949ec-572e-4e86-b860-b09b612085e8-tls-cert\" (UniqueName: \"kubernetes.io/secret/80d949ec-572e-4e86-b860-b09b612085e8-tls-cert\") pod \"80d949ec-572e-4e86-b860-b09b612085e8\" (UID: \"80d949ec-572e-4e86-b860-b09b612085e8\") "
Nov 04 07:32:43.072655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.072620    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d949ec-572e-4e86-b860-b09b612085e8-kube-api-access-zkp8f" pod "80d949ec-572e-4e86-b860-b09b612085e8" (UID: "80d949ec-572e-4e86-b860-b09b612085e8"). InnerVolumeSpecName "kube-api-access-zkp8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:32:43.080220 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.080196    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80d949ec-572e-4e86-b860-b09b612085e8-tls-cert" pod "80d949ec-572e-4e86-b860-b09b612085e8" (UID: "80d949ec-572e-4e86-b860-b09b612085e8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:32:43.171413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.171308    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d835348b-dfc8-4020-8bf1-5fe2b06c97f1-kube-api-access-mddn8\" (UniqueName: \"kubernetes.io/projected/d835348b-dfc8-4020-8bf1-5fe2b06c97f1-kube-api-access-mddn8\") pod \"d835348b-dfc8-4020-8bf1-5fe2b06c97f1\" (UID: \"d835348b-dfc8-4020-8bf1-5fe2b06c97f1\") "
Nov 04 07:32:43.171562 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.171554    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkp8f\" (UniqueName: \"kubernetes.io/projected/80d949ec-572e-4e86-b860-b09b612085e8-kube-api-access-zkp8f\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:32:43.171603 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.171566    5626 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80d949ec-572e-4e86-b860-b09b612085e8-tls-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:32:43.173338 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.173308    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d835348b-dfc8-4020-8bf1-5fe2b06c97f1-kube-api-access-mddn8" pod "d835348b-dfc8-4020-8bf1-5fe2b06c97f1" (UID: "d835348b-dfc8-4020-8bf1-5fe2b06c97f1"). InnerVolumeSpecName "kube-api-access-mddn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:32:43.272960 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.272923    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mddn8\" (UniqueName: \"kubernetes.io/projected/d835348b-dfc8-4020-8bf1-5fe2b06c97f1-kube-api-access-mddn8\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:32:43.585282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.585246    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-xgjgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Nov 04 07:32:43.585809 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.585307    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh" podUID="7ba40135-db97-4238-8bb8-f1fd581a1615" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Nov 04 07:32:43.656386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.655629    5626 generic.go:356] "Generic (PLEG): container finished" podID="d835348b-dfc8-4020-8bf1-5fe2b06c97f1" containerID="d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719" exitCode=0
Nov 04 07:32:43.656386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.655839    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-llg5c" event={"ID":"d835348b-dfc8-4020-8bf1-5fe2b06c97f1","Type":"ContainerDied","Data":"d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719"}
Nov 04 07:32:43.656386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.655882    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-llg5c" event={"ID":"d835348b-dfc8-4020-8bf1-5fe2b06c97f1","Type":"ContainerDied","Data":"adc93a730406648c992b37188926e77fc4b5a2c20f8e29010ff464406a7ce431"}
Nov 04 07:32:43.656386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.655912    5626 scope.go:122] "RemoveContainer" containerID="d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719"
Nov 04 07:32:43.656386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.656073    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-llg5c"
Nov 04 07:32:43.671804 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.671765    5626 generic.go:356] "Generic (PLEG): container finished" podID="80d949ec-572e-4e86-b860-b09b612085e8" containerID="206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c" exitCode=0
Nov 04 07:32:43.671953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.671832    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-4m66s" event={"ID":"80d949ec-572e-4e86-b860-b09b612085e8","Type":"ContainerDied","Data":"206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c"}
Nov 04 07:32:43.671953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.671866    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6889cbd8c4-4m66s" event={"ID":"80d949ec-572e-4e86-b860-b09b612085e8","Type":"ContainerDied","Data":"083be772122c6c2638ee3a66e2c6d8bec0f81d4e65857d4ca45a933861024661"}
Nov 04 07:32:43.671953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.671931    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-4m66s"
Nov 04 07:32:43.679053 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.679030    5626 scope.go:122] "RemoveContainer" containerID="d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719"
Nov 04 07:32:43.679418 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:32:43.679397    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719\": container with ID starting with d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719 not found: ID does not exist" containerID="d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719"
Nov 04 07:32:43.679502 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.679433    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719"} err="failed to get container status \"d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719\": rpc error: code = NotFound desc = could not find container \"d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719\": container with ID starting with d402c16864cd152e73d63111c2c613e3e1aec726a8984760c0c05ecced84f719 not found: ID does not exist"
Nov 04 07:32:43.679502 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.679456    5626 scope.go:122] "RemoveContainer" containerID="206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c"
Nov 04 07:32:43.687500 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.687483    5626 scope.go:122] "RemoveContainer" containerID="206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c"
Nov 04 07:32:43.687745 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:32:43.687730    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c\": container with ID starting with 206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c not found: ID does not exist" containerID="206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c"
Nov 04 07:32:43.687785 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.687750    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c"} err="failed to get container status \"206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c\": rpc error: code = NotFound desc = could not find container \"206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c\": container with ID starting with 206f896e1d4e8f269fa1a39be01042a242d46ef75bc14bb020dfc6f91bb92a9c not found: ID does not exist"
Nov 04 07:32:43.689885 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.689857    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-llg5c"]
Nov 04 07:32:43.698826 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.694972    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-llg5c"]
Nov 04 07:32:43.711105 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.711079    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-4m66s"]
Nov 04 07:32:43.715003 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:43.714980    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-4m66s"]
Nov 04 07:32:44.091612 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:44.091579    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80d949ec-572e-4e86-b860-b09b612085e8" path="/var/lib/kubelet/pods/80d949ec-572e-4e86-b860-b09b612085e8/volumes"
Nov 04 07:32:44.091956 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:44.091943    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d835348b-dfc8-4020-8bf1-5fe2b06c97f1" path="/var/lib/kubelet/pods/d835348b-dfc8-4020-8bf1-5fe2b06c97f1/volumes"
Nov 04 07:32:44.584901 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:44.584865    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-xgjgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Nov 04 07:32:44.585094 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:44.584918    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh" podUID="7ba40135-db97-4238-8bb8-f1fd581a1615" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Nov 04 07:32:45.638318 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:45.638289    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:32:45.638818 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:45.638756    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:32:45.639600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:32:45.639579    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh"
Nov 04 07:33:06.171221 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:33:06.171181    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:33:17.431558 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.431523    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-tzffp"]
Nov 04 07:33:17.432029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.431879    5626 cpu_manager.go:414] "containerMap: removing container" podUID="80d949ec-572e-4e86-b860-b09b612085e8" containerName="authorino"
Nov 04 07:33:17.432029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.431893    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80d949ec-572e-4e86-b860-b09b612085e8" containerName="authorino"
Nov 04 07:33:17.432029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.431905    5626 cpu_manager.go:414] "containerMap: removing container" podUID="d835348b-dfc8-4020-8bf1-5fe2b06c97f1" containerName="authorino"
Nov 04 07:33:17.432029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.431909    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d835348b-dfc8-4020-8bf1-5fe2b06c97f1" containerName="authorino"
Nov 04 07:33:17.432029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.431973    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="80d949ec-572e-4e86-b860-b09b612085e8" containerName="authorino"
Nov 04 07:33:17.432029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.431981    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="d835348b-dfc8-4020-8bf1-5fe2b06c97f1" containerName="authorino"
Nov 04 07:33:17.434807 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.434788    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp"
Nov 04 07:33:17.441600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.441581    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-c4f46\""
Nov 04 07:33:17.441710 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.441698    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 04 07:33:17.443192 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.443171    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Nov 04 07:33:17.447129 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.447105    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-tzffp"]
Nov 04 07:33:17.539199 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.539165    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjw9g\" (UniqueName: \"kubernetes.io/projected/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-kube-api-access-vjw9g\") pod \"ai-gateway-controller-5d8dddb676-tzffp\" (UID: \"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp"
Nov 04 07:33:17.539397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.539221    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-metrics-certs\") pod \"ai-gateway-controller-5d8dddb676-tzffp\" (UID: \"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp"
Nov 04 07:33:17.640302 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.640270    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjw9g\" (UniqueName: \"kubernetes.io/projected/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-kube-api-access-vjw9g\") pod \"ai-gateway-controller-5d8dddb676-tzffp\" (UID: \"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp"
Nov 04 07:33:17.640476 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.640325    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-metrics-certs\") pod \"ai-gateway-controller-5d8dddb676-tzffp\" (UID: \"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp"
Nov 04 07:33:17.642757 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.642740    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-metrics-certs\") pod \"ai-gateway-controller-5d8dddb676-tzffp\" (UID: \"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp"
Nov 04 07:33:17.652909 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.652886    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjw9g\" (UniqueName: \"kubernetes.io/projected/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-kube-api-access-vjw9g\") pod \"ai-gateway-controller-5d8dddb676-tzffp\" (UID: \"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp"
Nov 04 07:33:17.751087 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:17.751012    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp"
Nov 04 07:33:18.026137 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:18.026109    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-tzffp"]
Nov 04 07:33:18.028695 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:33:18.028666    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41527a1d_e546_4c8c_a4ef_0c8011f2d4e8.slice/crio-1845f7a8c020340c2101c37dcf9250f6a58a0fbab76b72a5e4d336c281d6942a WatchSource:0}: Error finding container 1845f7a8c020340c2101c37dcf9250f6a58a0fbab76b72a5e4d336c281d6942a: Status 404 returned error can't find the container with id 1845f7a8c020340c2101c37dcf9250f6a58a0fbab76b72a5e4d336c281d6942a
Nov 04 07:33:18.839559 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:18.839521    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp" event={"ID":"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8","Type":"ContainerStarted","Data":"1845f7a8c020340c2101c37dcf9250f6a58a0fbab76b72a5e4d336c281d6942a"}
Nov 04 07:33:20.455617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:20.455592    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 04 07:33:20.850836 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:20.850805    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp" event={"ID":"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8","Type":"ContainerStarted","Data":"c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87"}
Nov 04 07:33:20.850996 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:20.850937    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp"
Nov 04 07:33:20.872695 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:20.872644    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp" podStartSLOduration=1.4504895850000001 podStartE2EDuration="3.872632664s" totalImagesPullingTime="2.422143079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:33:20.870260903 +0000 UTC m=+1022.892476177" watchObservedRunningTime="2026-11-04 07:33:20.872632664 +0000 UTC m=+1022.894847956"
Nov 04 07:33:31.857382 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:31.857339    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp"
Nov 04 07:33:32.147173 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:32.147099    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"]
Nov 04 07:33:32.150635 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:32.150619    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"
Nov 04 07:33:32.164562 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:32.164538    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"]
Nov 04 07:33:32.275691 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:32.275650    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/507645ed-1327-42b2-98fe-f059d5e3eeea-metrics-certs\") pod \"ai-gateway-controller-84bbcb8bf9-hjnv7\" (UID: \"507645ed-1327-42b2-98fe-f059d5e3eeea\") " pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"
Nov 04 07:33:32.275894 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:32.275718    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sk86\" (UniqueName: \"kubernetes.io/projected/507645ed-1327-42b2-98fe-f059d5e3eeea-kube-api-access-7sk86\") pod \"ai-gateway-controller-84bbcb8bf9-hjnv7\" (UID: \"507645ed-1327-42b2-98fe-f059d5e3eeea\") " pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"
Nov 04 07:33:32.376282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:32.376252    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/507645ed-1327-42b2-98fe-f059d5e3eeea-metrics-certs\") pod \"ai-gateway-controller-84bbcb8bf9-hjnv7\" (UID: \"507645ed-1327-42b2-98fe-f059d5e3eeea\") " pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"
Nov 04 07:33:32.376449 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:32.376303    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sk86\" (UniqueName: \"kubernetes.io/projected/507645ed-1327-42b2-98fe-f059d5e3eeea-kube-api-access-7sk86\") pod \"ai-gateway-controller-84bbcb8bf9-hjnv7\" (UID: \"507645ed-1327-42b2-98fe-f059d5e3eeea\") " pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"
Nov 04 07:33:32.378887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:32.378862    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/507645ed-1327-42b2-98fe-f059d5e3eeea-metrics-certs\") pod \"ai-gateway-controller-84bbcb8bf9-hjnv7\" (UID: \"507645ed-1327-42b2-98fe-f059d5e3eeea\") " pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"
Nov 04 07:33:32.389306 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:32.389277    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sk86\" (UniqueName: \"kubernetes.io/projected/507645ed-1327-42b2-98fe-f059d5e3eeea-kube-api-access-7sk86\") pod \"ai-gateway-controller-84bbcb8bf9-hjnv7\" (UID: \"507645ed-1327-42b2-98fe-f059d5e3eeea\") " pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"
Nov 04 07:33:32.462301 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:32.462244    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"
Nov 04 07:33:32.735333 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:32.735308    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"]
Nov 04 07:33:32.737727 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:33:32.737698    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod507645ed_1327_42b2_98fe_f059d5e3eeea.slice/crio-58c08031c384d9be0c1b3aeea9d66089bbfa757cb37d09569264a77119799134 WatchSource:0}: Error finding container 58c08031c384d9be0c1b3aeea9d66089bbfa757cb37d09569264a77119799134: Status 404 returned error can't find the container with id 58c08031c384d9be0c1b3aeea9d66089bbfa757cb37d09569264a77119799134
Nov 04 07:33:32.911833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:32.911795    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7" event={"ID":"507645ed-1327-42b2-98fe-f059d5e3eeea","Type":"ContainerStarted","Data":"58c08031c384d9be0c1b3aeea9d66089bbfa757cb37d09569264a77119799134"}
Nov 04 07:33:33.918146 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:33.918112    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7" event={"ID":"507645ed-1327-42b2-98fe-f059d5e3eeea","Type":"ContainerStarted","Data":"07db9b83cd9eb615f14d0c47bc1d53454b6de0ae8094138df3ecc9ccf17b059c"}
Nov 04 07:33:33.918579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:33.918217    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"
Nov 04 07:33:33.938447 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:33.938399    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7" podStartSLOduration=1.551810572 podStartE2EDuration="1.938386128s" totalImagesPullingTime="386.575556ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:33:33.937259644 +0000 UTC m=+1035.959474904" watchObservedRunningTime="2026-11-04 07:33:33.938386128 +0000 UTC m=+1035.960601384"
Nov 04 07:33:36.171688 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:33:36.171649    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:33:44.924999 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:44.924967    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7"
Nov 04 07:33:44.977149 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:44.977110    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-tzffp"]
Nov 04 07:33:44.977351 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:44.977330    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp" podUID="41527a1d-e546-4c8c-a4ef-0c8011f2d4e8" containerName="manager" containerID="cri-o://c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87" gracePeriod=30
Nov 04 07:33:45.223628 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.223602    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp"
Nov 04 07:33:45.384924 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.384830    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-kube-api-access-vjw9g\" (UniqueName: \"kubernetes.io/projected/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-kube-api-access-vjw9g\") pod \"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8\" (UID: \"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8\") "
Nov 04 07:33:45.385126 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.385006    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-metrics-certs\") pod \"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8\" (UID: \"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8\") "
Nov 04 07:33:45.387030 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.386997    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-metrics-certs" pod "41527a1d-e546-4c8c-a4ef-0c8011f2d4e8" (UID: "41527a1d-e546-4c8c-a4ef-0c8011f2d4e8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:33:45.387168 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.387062    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-kube-api-access-vjw9g" pod "41527a1d-e546-4c8c-a4ef-0c8011f2d4e8" (UID: "41527a1d-e546-4c8c-a4ef-0c8011f2d4e8"). InnerVolumeSpecName "kube-api-access-vjw9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:33:45.486702 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.486630    5626 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-metrics-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:33:45.486702 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.486657    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjw9g\" (UniqueName: \"kubernetes.io/projected/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8-kube-api-access-vjw9g\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:33:45.975217 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.975179    5626 generic.go:356] "Generic (PLEG): container finished" podID="41527a1d-e546-4c8c-a4ef-0c8011f2d4e8" containerID="c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87" exitCode=0
Nov 04 07:33:45.975734 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.975242    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp"
Nov 04 07:33:45.975734 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.975261    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp" event={"ID":"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8","Type":"ContainerDied","Data":"c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87"}
Nov 04 07:33:45.975734 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.975310    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-tzffp" event={"ID":"41527a1d-e546-4c8c-a4ef-0c8011f2d4e8","Type":"ContainerDied","Data":"1845f7a8c020340c2101c37dcf9250f6a58a0fbab76b72a5e4d336c281d6942a"}
Nov 04 07:33:45.975734 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.975331    5626 scope.go:122] "RemoveContainer" containerID="c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87"
Nov 04 07:33:45.985009 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.984794    5626 scope.go:122] "RemoveContainer" containerID="c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87"
Nov 04 07:33:45.985108 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:33:45.985089    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87\": container with ID starting with c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87 not found: ID does not exist" containerID="c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87"
Nov 04 07:33:45.985149 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.985117    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87"} err="failed to get container status \"c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87\": rpc error: code = NotFound desc = could not find container \"c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87\": container with ID starting with c31af45776f6cae4be252b6b96a8c85ab42f3650ed7527d70adcb4533aac4e87 not found: ID does not exist"
Nov 04 07:33:45.998479 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:45.998458    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-tzffp"]
Nov 04 07:33:46.003435 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:46.003416    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-tzffp"]
Nov 04 07:33:46.090756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:46.090723    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41527a1d-e546-4c8c-a4ef-0c8011f2d4e8" path="/var/lib/kubelet/pods/41527a1d-e546-4c8c-a4ef-0c8011f2d4e8/volumes"
Nov 04 07:33:53.827711 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.827675    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"]
Nov 04 07:33:53.828295 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.828274    5626 cpu_manager.go:414] "containerMap: removing container" podUID="41527a1d-e546-4c8c-a4ef-0c8011f2d4e8" containerName="manager"
Nov 04 07:33:53.828388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.828297    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41527a1d-e546-4c8c-a4ef-0c8011f2d4e8" containerName="manager"
Nov 04 07:33:53.828499 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.828483    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="41527a1d-e546-4c8c-a4ef-0c8011f2d4e8" containerName="manager"
Nov 04 07:33:53.833667 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.833643    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.842405 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.842381    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 04 07:33:53.842543 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.842480    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 04 07:33:53.846502 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.846459    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-x7226\""
Nov 04 07:33:53.848455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.848425    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"]
Nov 04 07:33:53.849204 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.849189    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Nov 04 07:33:53.855736 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.855714    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fzzd\" (UniqueName: \"kubernetes.io/projected/f2fa15d8-82b1-4d8f-a541-574254719b50-kube-api-access-8fzzd\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.855845 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.855751    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fa15d8-82b1-4d8f-a541-574254719b50-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.855902 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.855880    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fa15d8-82b1-4d8f-a541-574254719b50-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.855964 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.855946    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fa15d8-82b1-4d8f-a541-574254719b50-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.856029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.856013    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fa15d8-82b1-4d8f-a541-574254719b50-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.856069 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.856055    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fa15d8-82b1-4d8f-a541-574254719b50-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.957000 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.956964    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fa15d8-82b1-4d8f-a541-574254719b50-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.957000 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.957010    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fa15d8-82b1-4d8f-a541-574254719b50-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.957266 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.957044    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fzzd\" (UniqueName: \"kubernetes.io/projected/f2fa15d8-82b1-4d8f-a541-574254719b50-kube-api-access-8fzzd\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.957266 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.957080    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fa15d8-82b1-4d8f-a541-574254719b50-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.957266 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.957135    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fa15d8-82b1-4d8f-a541-574254719b50-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.957266 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.957176    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fa15d8-82b1-4d8f-a541-574254719b50-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.957552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.957524    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fa15d8-82b1-4d8f-a541-574254719b50-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.957671 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.957591    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fa15d8-82b1-4d8f-a541-574254719b50-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.957862 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.957831    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fa15d8-82b1-4d8f-a541-574254719b50-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.959416 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.959393    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fa15d8-82b1-4d8f-a541-574254719b50-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.959587 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.959568    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fa15d8-82b1-4d8f-a541-574254719b50-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:53.971583 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:53.971561    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fzzd\" (UniqueName: \"kubernetes.io/projected/f2fa15d8-82b1-4d8f-a541-574254719b50-kube-api-access-8fzzd\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-v2t9p\" (UID: \"f2fa15d8-82b1-4d8f-a541-574254719b50\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:54.149026 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:54.148944    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:54.429893 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:54.429867    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"]
Nov 04 07:33:54.432112 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:33:54.432081    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2fa15d8_82b1_4d8f_a541_574254719b50.slice/crio-94fb576ee74c4e87ccb9fc6330a3440a698414d1dfd8fb4409c4a503ce902a0e WatchSource:0}: Error finding container 94fb576ee74c4e87ccb9fc6330a3440a698414d1dfd8fb4409c4a503ce902a0e: Status 404 returned error can't find the container with id 94fb576ee74c4e87ccb9fc6330a3440a698414d1dfd8fb4409c4a503ce902a0e
Nov 04 07:33:55.025007 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:55.024967    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p" event={"ID":"f2fa15d8-82b1-4d8f-a541-574254719b50","Type":"ContainerStarted","Data":"94fb576ee74c4e87ccb9fc6330a3440a698414d1dfd8fb4409c4a503ce902a0e"}
Nov 04 07:33:56.030504 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:56.030475    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p" event={"ID":"f2fa15d8-82b1-4d8f-a541-574254719b50","Type":"ContainerStarted","Data":"0ed6fbecb1acf2500edacd339e6f3df8d1d21c0e050d5b7d144b39d81e00cd89"}
Nov 04 07:33:56.052494 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:56.052446    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p" podStartSLOduration=1.824241395 podStartE2EDuration="3.052433361s" totalImagesPullingTime="1.228191966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:33:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:33:56.051866744 +0000 UTC m=+1058.074082004" watchObservedRunningTime="2026-11-04 07:33:56.052433361 +0000 UTC m=+1058.074648620"
Nov 04 07:33:56.149259 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:56.149222    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:33:57.093964 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.093930    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"]
Nov 04 07:33:57.098206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.098188    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.105972 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.105950    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Nov 04 07:33:57.107443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.107418    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"]
Nov 04 07:33:57.183947 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.183909    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d698e45f-e68d-4c6c-99d2-ab4a646437d0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.183947 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.183953    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkl5h\" (UniqueName: \"kubernetes.io/projected/d698e45f-e68d-4c6c-99d2-ab4a646437d0-kube-api-access-lkl5h\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.184155 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.184067    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d698e45f-e68d-4c6c-99d2-ab4a646437d0-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.184155 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.184095    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d698e45f-e68d-4c6c-99d2-ab4a646437d0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.184155 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.184130    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d698e45f-e68d-4c6c-99d2-ab4a646437d0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.184268 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.184216    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d698e45f-e68d-4c6c-99d2-ab4a646437d0-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.285274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.285235    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d698e45f-e68d-4c6c-99d2-ab4a646437d0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.285493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.285281    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkl5h\" (UniqueName: \"kubernetes.io/projected/d698e45f-e68d-4c6c-99d2-ab4a646437d0-kube-api-access-lkl5h\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.285493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.285341    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d698e45f-e68d-4c6c-99d2-ab4a646437d0-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.285493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.285385    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d698e45f-e68d-4c6c-99d2-ab4a646437d0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.285493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.285432    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d698e45f-e68d-4c6c-99d2-ab4a646437d0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.285493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.285461    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d698e45f-e68d-4c6c-99d2-ab4a646437d0-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.285740 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.285723    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d698e45f-e68d-4c6c-99d2-ab4a646437d0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.285803 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.285760    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d698e45f-e68d-4c6c-99d2-ab4a646437d0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.285855 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.285805    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d698e45f-e68d-4c6c-99d2-ab4a646437d0-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.287622 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.287599    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d698e45f-e68d-4c6c-99d2-ab4a646437d0-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.287907 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.287883    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d698e45f-e68d-4c6c-99d2-ab4a646437d0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.298319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.298295    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkl5h\" (UniqueName: \"kubernetes.io/projected/d698e45f-e68d-4c6c-99d2-ab4a646437d0-kube-api-access-lkl5h\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-226jt\" (UID: \"d698e45f-e68d-4c6c-99d2-ab4a646437d0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.412311 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.412223    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:33:57.691402 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.691373    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"]
Nov 04 07:33:57.693729 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:33:57.693701    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd698e45f_e68d_4c6c_99d2_ab4a646437d0.slice/crio-4a5deed273818c68a7a26f251e876b283ab1d0b0e816c96688f949d4f65073fb WatchSource:0}: Error finding container 4a5deed273818c68a7a26f251e876b283ab1d0b0e816c96688f949d4f65073fb: Status 404 returned error can't find the container with id 4a5deed273818c68a7a26f251e876b283ab1d0b0e816c96688f949d4f65073fb
Nov 04 07:33:57.848617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.848583    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"]
Nov 04 07:33:57.853381 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.853345    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.864178 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.864154    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Nov 04 07:33:57.865671 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.865599    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"]
Nov 04 07:33:57.892072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.891668    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d1867e7-cf85-4418-9a27-501735d097bd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.892072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.891740    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d1867e7-cf85-4418-9a27-501735d097bd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.892072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.891778    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fc6w\" (UniqueName: \"kubernetes.io/projected/5d1867e7-cf85-4418-9a27-501735d097bd-kube-api-access-7fc6w\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.892072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.891855    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d1867e7-cf85-4418-9a27-501735d097bd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.892072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.891894    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d1867e7-cf85-4418-9a27-501735d097bd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.892072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.891929    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d1867e7-cf85-4418-9a27-501735d097bd-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.992473 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.992449    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d1867e7-cf85-4418-9a27-501735d097bd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.992586 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.992499    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d1867e7-cf85-4418-9a27-501735d097bd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.992586 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.992539    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d1867e7-cf85-4418-9a27-501735d097bd-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.992586 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.992570    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d1867e7-cf85-4418-9a27-501735d097bd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.993239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.992626    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d1867e7-cf85-4418-9a27-501735d097bd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.993239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.992660    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fc6w\" (UniqueName: \"kubernetes.io/projected/5d1867e7-cf85-4418-9a27-501735d097bd-kube-api-access-7fc6w\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.993239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.992874    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d1867e7-cf85-4418-9a27-501735d097bd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.993239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.993113    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d1867e7-cf85-4418-9a27-501735d097bd-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.993239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.993149    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d1867e7-cf85-4418-9a27-501735d097bd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.995180 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.995159    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d1867e7-cf85-4418-9a27-501735d097bd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:57.995659 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:57.995638    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d1867e7-cf85-4418-9a27-501735d097bd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:58.042884 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:58.042853    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt" event={"ID":"d698e45f-e68d-4c6c-99d2-ab4a646437d0","Type":"ContainerStarted","Data":"4a5deed273818c68a7a26f251e876b283ab1d0b0e816c96688f949d4f65073fb"}
Nov 04 07:33:58.046412 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:58.046389    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fc6w\" (UniqueName: \"kubernetes.io/projected/5d1867e7-cf85-4418-9a27-501735d097bd-kube-api-access-7fc6w\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j\" (UID: \"5d1867e7-cf85-4418-9a27-501735d097bd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:58.169782 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:58.169750    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:58.451528 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:58.451410    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"]
Nov 04 07:33:58.454876 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:33:58.454826    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d1867e7_cf85_4418_9a27_501735d097bd.slice/crio-d85ba3899851416b559b703a7c892d1e37dd53e3e255dac8ec9250f7b98e3a6b WatchSource:0}: Error finding container d85ba3899851416b559b703a7c892d1e37dd53e3e255dac8ec9250f7b98e3a6b: Status 404 returned error can't find the container with id d85ba3899851416b559b703a7c892d1e37dd53e3e255dac8ec9250f7b98e3a6b
Nov 04 07:33:59.049554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:59.049496    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j" event={"ID":"5d1867e7-cf85-4418-9a27-501735d097bd","Type":"ContainerStarted","Data":"022d8df123900956f2f61bed7dda8621ed9b94b99f4f369b422a4a8f29cd25a7"}
Nov 04 07:33:59.049751 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:59.049607    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j" event={"ID":"5d1867e7-cf85-4418-9a27-501735d097bd","Type":"ContainerStarted","Data":"d85ba3899851416b559b703a7c892d1e37dd53e3e255dac8ec9250f7b98e3a6b"}
Nov 04 07:33:59.051387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:59.051340    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt" event={"ID":"d698e45f-e68d-4c6c-99d2-ab4a646437d0","Type":"ContainerStarted","Data":"590ecad2e27e9608aa762950a9831495bb9386ea4cca1449d4e01229ef27feb0"}
Nov 04 07:33:59.071546 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:59.071499    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j" podStartSLOduration=1.770189166 podStartE2EDuration="2.071485788s" totalImagesPullingTime="301.296622ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:33:59.070792986 +0000 UTC m=+1061.093008245" watchObservedRunningTime="2026-11-04 07:33:59.071485788 +0000 UTC m=+1061.093701074"
Nov 04 07:33:59.092192 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:59.092123    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt" podStartSLOduration=1.8089009520000001 podStartE2EDuration="2.092104144s" totalImagesPullingTime="283.203192ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:33:59.090982245 +0000 UTC m=+1061.113197502" watchObservedRunningTime="2026-11-04 07:33:59.092104144 +0000 UTC m=+1061.114319405"
Nov 04 07:33:59.170609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:59.170569    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:33:59.413062 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:33:59.412966    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:34:01.209386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.204289    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"]
Nov 04 07:34:01.211688 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.211658    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.222029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.221992    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Nov 04 07:34:01.222701 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.222676    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"]
Nov 04 07:34:01.324993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.324951    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/573aaebc-ca66-4bf2-84a5-2057a5e77b87-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.325139 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.324997    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smllk\" (UniqueName: \"kubernetes.io/projected/573aaebc-ca66-4bf2-84a5-2057a5e77b87-kube-api-access-smllk\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.325139 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.325102    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/573aaebc-ca66-4bf2-84a5-2057a5e77b87-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.325260 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.325141    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/573aaebc-ca66-4bf2-84a5-2057a5e77b87-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.325260 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.325160    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/573aaebc-ca66-4bf2-84a5-2057a5e77b87-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.325260 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.325234    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/573aaebc-ca66-4bf2-84a5-2057a5e77b87-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.426465 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.426426    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/573aaebc-ca66-4bf2-84a5-2057a5e77b87-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.426644 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.426477    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smllk\" (UniqueName: \"kubernetes.io/projected/573aaebc-ca66-4bf2-84a5-2057a5e77b87-kube-api-access-smllk\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.426644 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.426517    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/573aaebc-ca66-4bf2-84a5-2057a5e77b87-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.426644 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.426543    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/573aaebc-ca66-4bf2-84a5-2057a5e77b87-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.426830 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.426666    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/573aaebc-ca66-4bf2-84a5-2057a5e77b87-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.426830 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.426718    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/573aaebc-ca66-4bf2-84a5-2057a5e77b87-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.426939 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.426848    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/573aaebc-ca66-4bf2-84a5-2057a5e77b87-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.426939 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.426911    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/573aaebc-ca66-4bf2-84a5-2057a5e77b87-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.427057 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.427038    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/573aaebc-ca66-4bf2-84a5-2057a5e77b87-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.428695 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.428674    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/573aaebc-ca66-4bf2-84a5-2057a5e77b87-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.429112 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.429093    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/573aaebc-ca66-4bf2-84a5-2057a5e77b87-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.441625 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.441603    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smllk\" (UniqueName: \"kubernetes.io/projected/573aaebc-ca66-4bf2-84a5-2057a5e77b87-kube-api-access-smllk\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn\" (UID: \"573aaebc-ca66-4bf2-84a5-2057a5e77b87\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.526130 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.526037    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:01.835353 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:01.835327    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"]
Nov 04 07:34:01.837427 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:34:01.837396    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod573aaebc_ca66_4bf2_84a5_2057a5e77b87.slice/crio-f21227b70f763cbab55245b3256dcba1209d448858c42426e798e76d68999c57 WatchSource:0}: Error finding container f21227b70f763cbab55245b3256dcba1209d448858c42426e798e76d68999c57: Status 404 returned error can't find the container with id f21227b70f763cbab55245b3256dcba1209d448858c42426e798e76d68999c57
Nov 04 07:34:02.070125 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:02.070087    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" event={"ID":"573aaebc-ca66-4bf2-84a5-2057a5e77b87","Type":"ContainerStarted","Data":"f21227b70f763cbab55245b3256dcba1209d448858c42426e798e76d68999c57"}
Nov 04 07:34:03.077166 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:03.077127    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" event={"ID":"573aaebc-ca66-4bf2-84a5-2057a5e77b87","Type":"ContainerStarted","Data":"3da3ecdeeed15fdd291285d13009ec6a7458e9f8dc980c01ad50b890c5330dc2"}
Nov 04 07:34:03.104271 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:03.104211    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" podStartSLOduration=1.805539364 podStartE2EDuration="2.104194729s" totalImagesPullingTime="298.655365ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:03.101941392 +0000 UTC m=+1065.124156653" watchObservedRunningTime="2026-11-04 07:34:03.104194729 +0000 UTC m=+1065.126409989"
Nov 04 07:34:03.527092 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:03.527006    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:04.150182 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:04.150142    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:34:04.162796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:04.162772    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:34:05.101880 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:05.101849    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p"
Nov 04 07:34:06.172111 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:34:06.172073    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:34:07.412468 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:07.412436    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:34:07.424919 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:07.424891    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:34:08.115327 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:08.115302    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt"
Nov 04 07:34:08.170933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:08.170905    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:34:08.183433 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:08.183408    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:34:09.119183 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:09.119153    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j"
Nov 04 07:34:11.526321 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:11.526285    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:11.538748 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:11.538722    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:12.139967 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:12.139934    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:34:14.031434 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.031399    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7cc7fd6fd6-lmgkl"]
Nov 04 07:34:14.035009 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.034989    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cc7fd6fd6-lmgkl"
Nov 04 07:34:14.041105 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.041078    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c9nbb\""
Nov 04 07:34:14.041105 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.041091    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Nov 04 07:34:14.041292 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.041187    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 07:34:14.042933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.042912    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 07:34:14.045847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.045820    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7cc7fd6fd6-lmgkl"]
Nov 04 07:34:14.145552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.145513    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/84888547-b6a2-4dc5-9a93-ac851db37e0e-tls-cert\") pod \"authorino-7cc7fd6fd6-lmgkl\" (UID: \"84888547-b6a2-4dc5-9a93-ac851db37e0e\") " pod="kuadrant-system/authorino-7cc7fd6fd6-lmgkl"
Nov 04 07:34:14.145762 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.145682    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmqn8\" (UniqueName: \"kubernetes.io/projected/84888547-b6a2-4dc5-9a93-ac851db37e0e-kube-api-access-vmqn8\") pod \"authorino-7cc7fd6fd6-lmgkl\" (UID: \"84888547-b6a2-4dc5-9a93-ac851db37e0e\") " pod="kuadrant-system/authorino-7cc7fd6fd6-lmgkl"
Nov 04 07:34:14.246968 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.246926    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmqn8\" (UniqueName: \"kubernetes.io/projected/84888547-b6a2-4dc5-9a93-ac851db37e0e-kube-api-access-vmqn8\") pod \"authorino-7cc7fd6fd6-lmgkl\" (UID: \"84888547-b6a2-4dc5-9a93-ac851db37e0e\") " pod="kuadrant-system/authorino-7cc7fd6fd6-lmgkl"
Nov 04 07:34:14.246968 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.246969    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/84888547-b6a2-4dc5-9a93-ac851db37e0e-tls-cert\") pod \"authorino-7cc7fd6fd6-lmgkl\" (UID: \"84888547-b6a2-4dc5-9a93-ac851db37e0e\") " pod="kuadrant-system/authorino-7cc7fd6fd6-lmgkl"
Nov 04 07:34:14.249463 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.249437    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/84888547-b6a2-4dc5-9a93-ac851db37e0e-tls-cert\") pod \"authorino-7cc7fd6fd6-lmgkl\" (UID: \"84888547-b6a2-4dc5-9a93-ac851db37e0e\") " pod="kuadrant-system/authorino-7cc7fd6fd6-lmgkl"
Nov 04 07:34:14.287308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.287224    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmqn8\" (UniqueName: \"kubernetes.io/projected/84888547-b6a2-4dc5-9a93-ac851db37e0e-kube-api-access-vmqn8\") pod \"authorino-7cc7fd6fd6-lmgkl\" (UID: \"84888547-b6a2-4dc5-9a93-ac851db37e0e\") " pod="kuadrant-system/authorino-7cc7fd6fd6-lmgkl"
Nov 04 07:34:14.347535 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.347495    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cc7fd6fd6-lmgkl"
Nov 04 07:34:14.623102 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:14.623078    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7cc7fd6fd6-lmgkl"]
Nov 04 07:34:14.625486 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:34:14.625448    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84888547_b6a2_4dc5_9a93_ac851db37e0e.slice/crio-b085f2333c21ba1539b776356545dbea22763b337861103c940133a2ac94b1f0 WatchSource:0}: Error finding container b085f2333c21ba1539b776356545dbea22763b337861103c940133a2ac94b1f0: Status 404 returned error can't find the container with id b085f2333c21ba1539b776356545dbea22763b337861103c940133a2ac94b1f0
Nov 04 07:34:15.152501 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.152470    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cc7fd6fd6-lmgkl" event={"ID":"84888547-b6a2-4dc5-9a93-ac851db37e0e","Type":"ContainerStarted","Data":"b085f2333c21ba1539b776356545dbea22763b337861103c940133a2ac94b1f0"}
Nov 04 07:34:15.703602 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.703567    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"]
Nov 04 07:34:15.707439 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.707390    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.729089 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.729034    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"]
Nov 04 07:34:15.760806 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.760773    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.761010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.760840    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.761010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.760924    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.761010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.760958    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.761010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.760977    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.761222 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.761047    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.761222 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.761089    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.761222 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.761176    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q79n\" (UniqueName: \"kubernetes.io/projected/71b5a865-3278-44ff-a614-7c6c4d0c42c5-kube-api-access-5q79n\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.761392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.761243    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862039 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.861958    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862039 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862006    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862039 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862026    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862281 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862043    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862281 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862061    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862281 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862078    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862281 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862112    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q79n\" (UniqueName: \"kubernetes.io/projected/71b5a865-3278-44ff-a614-7c6c4d0c42c5-kube-api-access-5q79n\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862281 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862150    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862281 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862200    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862606 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862492    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862660 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862621    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862725 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862703    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862843 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862825    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.862918 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.862901    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.864461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.864437    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.864781 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.864761    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.878688 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.878665    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71b5a865-3278-44ff-a614-7c6c4d0c42c5-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:15.879979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:15.879963    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q79n\" (UniqueName: \"kubernetes.io/projected/71b5a865-3278-44ff-a614-7c6c4d0c42c5-kube-api-access-5q79n\") pod \"maas-default-gateway-openshift-default-74854d8ccf-rxf9k\" (UID: \"71b5a865-3278-44ff-a614-7c6c4d0c42c5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:16.023780 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:16.023743    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:16.159386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:16.159266    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cc7fd6fd6-lmgkl" event={"ID":"84888547-b6a2-4dc5-9a93-ac851db37e0e","Type":"ContainerStarted","Data":"c2c1573ce71b4fb1c9ab0f9ed5c6e61807a48d8af058724c53acc857efc16400"}
Nov 04 07:34:16.179110 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:16.179051    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-7cc7fd6fd6-lmgkl" podStartSLOduration=1.239342185 podStartE2EDuration="2.179034736s" totalImagesPullingTime="939.692551ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:16.177595399 +0000 UTC m=+1078.199810656" watchObservedRunningTime="2026-11-04 07:34:16.179034736 +0000 UTC m=+1078.201250042"
Nov 04 07:34:16.312221 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:16.312186    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"]
Nov 04 07:34:16.313740 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:34:16.313713    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b5a865_3278_44ff_a614_7c6c4d0c42c5.slice/crio-732791cec1283904bf1a3904a406262d438341d3731212f87fdaeba9e6db940f WatchSource:0}: Error finding container 732791cec1283904bf1a3904a406262d438341d3731212f87fdaeba9e6db940f: Status 404 returned error can't find the container with id 732791cec1283904bf1a3904a406262d438341d3731212f87fdaeba9e6db940f
Nov 04 07:34:16.316203 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:16.316164    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:34:16.316302 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:16.316250    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:34:16.316353 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:16.316301    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:34:17.174725 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:17.174620    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k" event={"ID":"71b5a865-3278-44ff-a614-7c6c4d0c42c5","Type":"ContainerStarted","Data":"98932625c63faadba898bc14a14fe522996aae4efec3c2528815db43d66e821f"}
Nov 04 07:34:17.174725 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:17.174680    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k" event={"ID":"71b5a865-3278-44ff-a614-7c6c4d0c42c5","Type":"ContainerStarted","Data":"732791cec1283904bf1a3904a406262d438341d3731212f87fdaeba9e6db940f"}
Nov 04 07:34:17.324704 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:17.324637    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k" podStartSLOduration=2.324618974 podStartE2EDuration="2.324618974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:17.311242585 +0000 UTC m=+1079.333457856" watchObservedRunningTime="2026-11-04 07:34:17.324618974 +0000 UTC m=+1079.346834234"
Nov 04 07:34:18.024687 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:18.024648    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:18.026321 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:18.026288    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-rxf9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 04 07:34:18.026493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:18.026336    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k" podUID="71b5a865-3278-44ff-a614-7c6c4d0c42c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 04 07:34:19.025075 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:19.025038    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-rxf9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 04 07:34:19.025464 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:19.025125    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k" podUID="71b5a865-3278-44ff-a614-7c6c4d0c42c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 04 07:34:20.024699 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:20.024661    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-rxf9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 04 07:34:20.024875 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:20.024738    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k" podUID="71b5a865-3278-44ff-a614-7c6c4d0c42c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 04 07:34:21.025053 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:21.025016    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-rxf9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 04 07:34:21.025429 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:21.025065    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k" podUID="71b5a865-3278-44ff-a614-7c6c4d0c42c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 04 07:34:22.025171 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:22.025130    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-rxf9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 04 07:34:22.025664 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:22.025214    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k" podUID="71b5a865-3278-44ff-a614-7c6c4d0c42c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 04 07:34:23.025291 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:23.025251    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-rxf9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Nov 04 07:34:23.025715 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:23.025315    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k" podUID="71b5a865-3278-44ff-a614-7c6c4d0c42c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Nov 04 07:34:24.864458 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:24.864424    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:24.865045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:24.864882    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:24.865727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:24.865706    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k"
Nov 04 07:34:36.173298 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:34:36.173212    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:34:49.026943 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.026909    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"]
Nov 04 07:34:49.030657 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.030641    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"
Nov 04 07:34:49.037766 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.037745    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-1f6ce4-dockercfg-gm2vd\""
Nov 04 07:34:49.038422 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.038406    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-1f6ce4\""
Nov 04 07:34:49.048251 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.048217    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"]
Nov 04 07:34:49.072698 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.072666    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"]
Nov 04 07:34:49.076240 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.076216    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:34:49.079651 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:34:49.079615    5626 status_manager.go:1045] "Failed to get status for pod" err="pods \"payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn\" is forbidden: User \"system:node:ip-10-0-142-197.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-197.us-east-2.compute.internal' and this object" podUID="41718dd1-af1a-4e5b-953a-f9734b3f3483" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:34:49.135670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.135638    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"]
Nov 04 07:34:49.138013 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.137983    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmxtm\" (UniqueName: \"kubernetes.io/projected/41718dd1-af1a-4e5b-953a-f9734b3f3483-kube-api-access-xmxtm\") pod \"payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn\" (UID: \"41718dd1-af1a-4e5b-953a-f9734b3f3483\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:34:49.138176 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.138041    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41718dd1-af1a-4e5b-953a-f9734b3f3483-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn\" (UID: \"41718dd1-af1a-4e5b-953a-f9734b3f3483\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:34:49.138176 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.138142    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxwct\" (UniqueName: \"kubernetes.io/projected/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-kube-api-access-wxwct\") pod \"payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm\" (UID: \"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"
Nov 04 07:34:49.138284 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.138187    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm\" (UID: \"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"
Nov 04 07:34:49.238867 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.238823    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm\" (UID: \"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"
Nov 04 07:34:49.239378 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.239327    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmxtm\" (UniqueName: \"kubernetes.io/projected/41718dd1-af1a-4e5b-953a-f9734b3f3483-kube-api-access-xmxtm\") pod \"payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn\" (UID: \"41718dd1-af1a-4e5b-953a-f9734b3f3483\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:34:49.239638 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.239616    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41718dd1-af1a-4e5b-953a-f9734b3f3483-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn\" (UID: \"41718dd1-af1a-4e5b-953a-f9734b3f3483\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:34:49.239845 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.239813    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxwct\" (UniqueName: \"kubernetes.io/projected/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-kube-api-access-wxwct\") pod \"payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm\" (UID: \"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"
Nov 04 07:34:49.240174 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.240146    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm\" (UID: \"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"
Nov 04 07:34:49.240438 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.240420    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41718dd1-af1a-4e5b-953a-f9734b3f3483-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn\" (UID: \"41718dd1-af1a-4e5b-953a-f9734b3f3483\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:34:49.285919 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.285886    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmxtm\" (UniqueName: \"kubernetes.io/projected/41718dd1-af1a-4e5b-953a-f9734b3f3483-kube-api-access-xmxtm\") pod \"payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn\" (UID: \"41718dd1-af1a-4e5b-953a-f9734b3f3483\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:34:49.288127 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.288104    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxwct\" (UniqueName: \"kubernetes.io/projected/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-kube-api-access-wxwct\") pod \"payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm\" (UID: \"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"
Nov 04 07:34:49.343857 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.343813    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"
Nov 04 07:34:49.388743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.388705    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:34:49.625448 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.625418    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"]
Nov 04 07:34:49.628092 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:34:49.628062    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e98a2b_fc18_4ec7_9fc0_34330c4fc120.slice/crio-ffc24a7fec303654e8eae315d84793356521e47d50c197c04d39958314f23bc3 WatchSource:0}: Error finding container ffc24a7fec303654e8eae315d84793356521e47d50c197c04d39958314f23bc3: Status 404 returned error can't find the container with id ffc24a7fec303654e8eae315d84793356521e47d50c197c04d39958314f23bc3
Nov 04 07:34:49.675984 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.675956    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"]
Nov 04 07:34:49.681407 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:34:49.678261    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41718dd1_af1a_4e5b_953a_f9734b3f3483.slice/crio-6d44aede2ddd89987e078f5bd8461bf475bfe490c05d0105656260df81718156 WatchSource:0}: Error finding container 6d44aede2ddd89987e078f5bd8461bf475bfe490c05d0105656260df81718156: Status 404 returned error can't find the container with id 6d44aede2ddd89987e078f5bd8461bf475bfe490c05d0105656260df81718156
Nov 04 07:34:49.934353 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.934275    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"]
Nov 04 07:34:49.938962 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.938945    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"
Nov 04 07:34:49.953172 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.953143    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"]
Nov 04 07:34:49.953519 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.953500    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-016e6c\""
Nov 04 07:34:49.970229 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.970198    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"]
Nov 04 07:34:49.973661 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.973638    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"
Nov 04 07:34:49.984672 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.984549    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"]
Nov 04 07:34:49.987040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:49.987017    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-016e6c-dockercfg-4pqhp\""
Nov 04 07:34:50.049475 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.049436    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz4m9\" (UniqueName: \"kubernetes.io/projected/7244c03f-1118-4bab-92ec-34894a8c9c5f-kube-api-access-mz4m9\") pod \"payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh\" (UID: \"7244c03f-1118-4bab-92ec-34894a8c9c5f\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"
Nov 04 07:34:50.050045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.049661    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efa49e6f-5463-4724-a2b6-51882cb8b317-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc\" (UID: \"efa49e6f-5463-4724-a2b6-51882cb8b317\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"
Nov 04 07:34:50.050045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.049764    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xknsb\" (UniqueName: \"kubernetes.io/projected/efa49e6f-5463-4724-a2b6-51882cb8b317-kube-api-access-xknsb\") pod \"payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc\" (UID: \"efa49e6f-5463-4724-a2b6-51882cb8b317\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"
Nov 04 07:34:50.050045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.049817    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7244c03f-1118-4bab-92ec-34894a8c9c5f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh\" (UID: \"7244c03f-1118-4bab-92ec-34894a8c9c5f\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"
Nov 04 07:34:50.150785 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.150750    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efa49e6f-5463-4724-a2b6-51882cb8b317-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc\" (UID: \"efa49e6f-5463-4724-a2b6-51882cb8b317\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"
Nov 04 07:34:50.150785 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.150789    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xknsb\" (UniqueName: \"kubernetes.io/projected/efa49e6f-5463-4724-a2b6-51882cb8b317-kube-api-access-xknsb\") pod \"payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc\" (UID: \"efa49e6f-5463-4724-a2b6-51882cb8b317\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"
Nov 04 07:34:50.151021 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.150809    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7244c03f-1118-4bab-92ec-34894a8c9c5f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh\" (UID: \"7244c03f-1118-4bab-92ec-34894a8c9c5f\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"
Nov 04 07:34:50.151021 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.150852    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz4m9\" (UniqueName: \"kubernetes.io/projected/7244c03f-1118-4bab-92ec-34894a8c9c5f-kube-api-access-mz4m9\") pod \"payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh\" (UID: \"7244c03f-1118-4bab-92ec-34894a8c9c5f\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"
Nov 04 07:34:50.151538 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.151510    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efa49e6f-5463-4724-a2b6-51882cb8b317-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc\" (UID: \"efa49e6f-5463-4724-a2b6-51882cb8b317\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"
Nov 04 07:34:50.151794 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.151646    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7244c03f-1118-4bab-92ec-34894a8c9c5f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh\" (UID: \"7244c03f-1118-4bab-92ec-34894a8c9c5f\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"
Nov 04 07:34:50.169048 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.169015    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xknsb\" (UniqueName: \"kubernetes.io/projected/efa49e6f-5463-4724-a2b6-51882cb8b317-kube-api-access-xknsb\") pod \"payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc\" (UID: \"efa49e6f-5463-4724-a2b6-51882cb8b317\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"
Nov 04 07:34:50.171457 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.171427    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz4m9\" (UniqueName: \"kubernetes.io/projected/7244c03f-1118-4bab-92ec-34894a8c9c5f-kube-api-access-mz4m9\") pod \"payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh\" (UID: \"7244c03f-1118-4bab-92ec-34894a8c9c5f\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"
Nov 04 07:34:50.252684 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.252591    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"
Nov 04 07:34:50.289401 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.289176    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"
Nov 04 07:34:50.404386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.404275    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm" event={"ID":"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120","Type":"ContainerStarted","Data":"ffc24a7fec303654e8eae315d84793356521e47d50c197c04d39958314f23bc3"}
Nov 04 07:34:50.408989 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.408909    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn" event={"ID":"41718dd1-af1a-4e5b-953a-f9734b3f3483","Type":"ContainerStarted","Data":"6d44aede2ddd89987e078f5bd8461bf475bfe490c05d0105656260df81718156"}
Nov 04 07:34:50.542258 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.542221    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"]
Nov 04 07:34:50.547034 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.547006    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"
Nov 04 07:34:50.561660 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.561632    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-60ed4a-dockercfg-jwt2t\""
Nov 04 07:34:50.561806 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.561723    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-60ed4a\""
Nov 04 07:34:50.568156 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.568036    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"]
Nov 04 07:34:50.570667 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.570643    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"]
Nov 04 07:34:50.583352 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:34:50.582997    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa49e6f_5463_4724_a2b6_51882cb8b317.slice/crio-e3c902f0d0484f4244a8949ca730f16ab462753081bd2425e34660eab426623e WatchSource:0}: Error finding container e3c902f0d0484f4244a8949ca730f16ab462753081bd2425e34660eab426623e: Status 404 returned error can't find the container with id e3c902f0d0484f4244a8949ca730f16ab462753081bd2425e34660eab426623e
Nov 04 07:34:50.588100 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.588070    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"]
Nov 04 07:34:50.592814 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.592794    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"
Nov 04 07:34:50.603710 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.603680    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"]
Nov 04 07:34:50.634099 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.634072    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"]
Nov 04 07:34:50.647687 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:34:50.638437    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7244c03f_1118_4bab_92ec_34894a8c9c5f.slice/crio-9e3b6b6611a956cb388ed1fbf46dcc4bd30e59168cc8536b357334329e7d1eb4 WatchSource:0}: Error finding container 9e3b6b6611a956cb388ed1fbf46dcc4bd30e59168cc8536b357334329e7d1eb4: Status 404 returned error can't find the container with id 9e3b6b6611a956cb388ed1fbf46dcc4bd30e59168cc8536b357334329e7d1eb4
Nov 04 07:34:50.657101 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.657058    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j79lg\" (UniqueName: \"kubernetes.io/projected/9b8fb283-b77c-4758-9497-84ba24890da2-kube-api-access-j79lg\") pod \"payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt\" (UID: \"9b8fb283-b77c-4758-9497-84ba24890da2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"
Nov 04 07:34:50.657229 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.657126    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8fb283-b77c-4758-9497-84ba24890da2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt\" (UID: \"9b8fb283-b77c-4758-9497-84ba24890da2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"
Nov 04 07:34:50.657229 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.657159    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc7cc\" (UniqueName: \"kubernetes.io/projected/a8b6a0dc-8600-4ab9-923f-3d680473f39f-kube-api-access-sc7cc\") pod \"payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k\" (UID: \"a8b6a0dc-8600-4ab9-923f-3d680473f39f\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"
Nov 04 07:34:50.657229 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.657222    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b6a0dc-8600-4ab9-923f-3d680473f39f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k\" (UID: \"a8b6a0dc-8600-4ab9-923f-3d680473f39f\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"
Nov 04 07:34:50.758333 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.758299    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b6a0dc-8600-4ab9-923f-3d680473f39f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k\" (UID: \"a8b6a0dc-8600-4ab9-923f-3d680473f39f\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"
Nov 04 07:34:50.758548 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.758416    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j79lg\" (UniqueName: \"kubernetes.io/projected/9b8fb283-b77c-4758-9497-84ba24890da2-kube-api-access-j79lg\") pod \"payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt\" (UID: \"9b8fb283-b77c-4758-9497-84ba24890da2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"
Nov 04 07:34:50.758548 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.758450    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8fb283-b77c-4758-9497-84ba24890da2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt\" (UID: \"9b8fb283-b77c-4758-9497-84ba24890da2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"
Nov 04 07:34:50.758548 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.758470    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc7cc\" (UniqueName: \"kubernetes.io/projected/a8b6a0dc-8600-4ab9-923f-3d680473f39f-kube-api-access-sc7cc\") pod \"payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k\" (UID: \"a8b6a0dc-8600-4ab9-923f-3d680473f39f\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"
Nov 04 07:34:50.759522 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.759489    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b6a0dc-8600-4ab9-923f-3d680473f39f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k\" (UID: \"a8b6a0dc-8600-4ab9-923f-3d680473f39f\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"
Nov 04 07:34:50.760459 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.760434    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8fb283-b77c-4758-9497-84ba24890da2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt\" (UID: \"9b8fb283-b77c-4758-9497-84ba24890da2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"
Nov 04 07:34:50.777533 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.777503    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc7cc\" (UniqueName: \"kubernetes.io/projected/a8b6a0dc-8600-4ab9-923f-3d680473f39f-kube-api-access-sc7cc\") pod \"payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k\" (UID: \"a8b6a0dc-8600-4ab9-923f-3d680473f39f\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"
Nov 04 07:34:50.784655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.784628    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j79lg\" (UniqueName: \"kubernetes.io/projected/9b8fb283-b77c-4758-9497-84ba24890da2-kube-api-access-j79lg\") pod \"payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt\" (UID: \"9b8fb283-b77c-4758-9497-84ba24890da2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"
Nov 04 07:34:50.865247 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.865163    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"
Nov 04 07:34:50.923178 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:50.923136    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"
Nov 04 07:34:51.173716 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.173289    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"]
Nov 04 07:34:51.179422 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:34:51.179356    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8fb283_b77c_4758_9497_84ba24890da2.slice/crio-5b35f678cf63ec3331170fb304116baeb8f5188f97be09f6b0afb8e879054f85 WatchSource:0}: Error finding container 5b35f678cf63ec3331170fb304116baeb8f5188f97be09f6b0afb8e879054f85: Status 404 returned error can't find the container with id 5b35f678cf63ec3331170fb304116baeb8f5188f97be09f6b0afb8e879054f85
Nov 04 07:34:51.239412 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.239374    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"]
Nov 04 07:34:51.249686 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:34:51.249655    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b6a0dc_8600_4ab9_923f_3d680473f39f.slice/crio-bb33752bae1be2f5017ca01162ce3a59ce76f7b6ee41cf944066c5a0b0338f06 WatchSource:0}: Error finding container bb33752bae1be2f5017ca01162ce3a59ce76f7b6ee41cf944066c5a0b0338f06: Status 404 returned error can't find the container with id bb33752bae1be2f5017ca01162ce3a59ce76f7b6ee41cf944066c5a0b0338f06
Nov 04 07:34:51.420723 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.420689    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc" event={"ID":"efa49e6f-5463-4724-a2b6-51882cb8b317","Type":"ContainerStarted","Data":"e3c902f0d0484f4244a8949ca730f16ab462753081bd2425e34660eab426623e"}
Nov 04 07:34:51.423033 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.423000    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt" event={"ID":"9b8fb283-b77c-4758-9497-84ba24890da2","Type":"ContainerStarted","Data":"5b35f678cf63ec3331170fb304116baeb8f5188f97be09f6b0afb8e879054f85"}
Nov 04 07:34:51.425159 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.425098    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh" event={"ID":"7244c03f-1118-4bab-92ec-34894a8c9c5f","Type":"ContainerStarted","Data":"9e3b6b6611a956cb388ed1fbf46dcc4bd30e59168cc8536b357334329e7d1eb4"}
Nov 04 07:34:51.427000 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.426975    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k" event={"ID":"a8b6a0dc-8600-4ab9-923f-3d680473f39f","Type":"ContainerStarted","Data":"bb33752bae1be2f5017ca01162ce3a59ce76f7b6ee41cf944066c5a0b0338f06"}
Nov 04 07:34:51.663389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.663204    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"]
Nov 04 07:34:51.667744 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.667584    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"
Nov 04 07:34:51.671902 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.671875    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"]
Nov 04 07:34:51.674584 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.674519    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-df3be8\""
Nov 04 07:34:51.674933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.674913    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-df3be8-dockercfg-295bg\""
Nov 04 07:34:51.676780 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.676014    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"
Nov 04 07:34:51.681882 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.681859    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"]
Nov 04 07:34:51.690907 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.690877    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"]
Nov 04 07:34:51.772024 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.771962    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8thbh\" (UniqueName: \"kubernetes.io/projected/3477aa8d-a28d-4e96-8a17-1f177c1080ff-kube-api-access-8thbh\") pod \"payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b\" (UID: \"3477aa8d-a28d-4e96-8a17-1f177c1080ff\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"
Nov 04 07:34:51.772218 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.772072    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3477aa8d-a28d-4e96-8a17-1f177c1080ff-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b\" (UID: \"3477aa8d-a28d-4e96-8a17-1f177c1080ff\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"
Nov 04 07:34:51.772218 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.772107    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kg4x\" (UniqueName: \"kubernetes.io/projected/9661dd1c-40ae-420f-96cf-af9a01477333-kube-api-access-8kg4x\") pod \"payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv\" (UID: \"9661dd1c-40ae-420f-96cf-af9a01477333\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"
Nov 04 07:34:51.772627 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.772439    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9661dd1c-40ae-420f-96cf-af9a01477333-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv\" (UID: \"9661dd1c-40ae-420f-96cf-af9a01477333\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"
Nov 04 07:34:51.874143 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.874094    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9661dd1c-40ae-420f-96cf-af9a01477333-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv\" (UID: \"9661dd1c-40ae-420f-96cf-af9a01477333\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"
Nov 04 07:34:51.874306 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.874177    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8thbh\" (UniqueName: \"kubernetes.io/projected/3477aa8d-a28d-4e96-8a17-1f177c1080ff-kube-api-access-8thbh\") pod \"payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b\" (UID: \"3477aa8d-a28d-4e96-8a17-1f177c1080ff\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"
Nov 04 07:34:51.874306 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.874236    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3477aa8d-a28d-4e96-8a17-1f177c1080ff-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b\" (UID: \"3477aa8d-a28d-4e96-8a17-1f177c1080ff\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"
Nov 04 07:34:51.874306 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.874265    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kg4x\" (UniqueName: \"kubernetes.io/projected/9661dd1c-40ae-420f-96cf-af9a01477333-kube-api-access-8kg4x\") pod \"payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv\" (UID: \"9661dd1c-40ae-420f-96cf-af9a01477333\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"
Nov 04 07:34:51.875578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.875470    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9661dd1c-40ae-420f-96cf-af9a01477333-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv\" (UID: \"9661dd1c-40ae-420f-96cf-af9a01477333\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"
Nov 04 07:34:51.875578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.875491    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3477aa8d-a28d-4e96-8a17-1f177c1080ff-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b\" (UID: \"3477aa8d-a28d-4e96-8a17-1f177c1080ff\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"
Nov 04 07:34:51.891092 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.889195    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8thbh\" (UniqueName: \"kubernetes.io/projected/3477aa8d-a28d-4e96-8a17-1f177c1080ff-kube-api-access-8thbh\") pod \"payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b\" (UID: \"3477aa8d-a28d-4e96-8a17-1f177c1080ff\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"
Nov 04 07:34:51.893500 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.893396    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kg4x\" (UniqueName: \"kubernetes.io/projected/9661dd1c-40ae-420f-96cf-af9a01477333-kube-api-access-8kg4x\") pod \"payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv\" (UID: \"9661dd1c-40ae-420f-96cf-af9a01477333\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"
Nov 04 07:34:51.999658 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:51.999572    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"
Nov 04 07:34:52.034211 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:52.034178    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"
Nov 04 07:34:52.348290 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:52.348248    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"]
Nov 04 07:34:52.353936 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:34:52.353905    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9661dd1c_40ae_420f_96cf_af9a01477333.slice/crio-1742923402cce20ae78658b196fc1cced92580eb2f9c33772990d777a0e0c036 WatchSource:0}: Error finding container 1742923402cce20ae78658b196fc1cced92580eb2f9c33772990d777a0e0c036: Status 404 returned error can't find the container with id 1742923402cce20ae78658b196fc1cced92580eb2f9c33772990d777a0e0c036
Nov 04 07:34:52.389694 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:52.389658    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"]
Nov 04 07:34:52.394860 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:34:52.394613    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3477aa8d_a28d_4e96_8a17_1f177c1080ff.slice/crio-b76cd21176df16df5c7c43e3da100ec1239f3173d7da3d6718471fa883a815e9 WatchSource:0}: Error finding container b76cd21176df16df5c7c43e3da100ec1239f3173d7da3d6718471fa883a815e9: Status 404 returned error can't find the container with id b76cd21176df16df5c7c43e3da100ec1239f3173d7da3d6718471fa883a815e9
Nov 04 07:34:52.439569 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:52.439526    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv" event={"ID":"9661dd1c-40ae-420f-96cf-af9a01477333","Type":"ContainerStarted","Data":"1742923402cce20ae78658b196fc1cced92580eb2f9c33772990d777a0e0c036"}
Nov 04 07:34:52.443657 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:52.443581    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b" event={"ID":"3477aa8d-a28d-4e96-8a17-1f177c1080ff","Type":"ContainerStarted","Data":"b76cd21176df16df5c7c43e3da100ec1239f3173d7da3d6718471fa883a815e9"}
Nov 04 07:34:53.959516 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:53.956459    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"]
Nov 04 07:34:53.968070 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:53.968040    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:53.977096 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:53.977036    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"]
Nov 04 07:34:53.978341 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:53.978314    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-29d3ee-openshift-default-dockercfg-xktsc\""
Nov 04 07:34:54.123413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.113142    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-certs\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.123413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.113184    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-socket\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.123413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.113206    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-token\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.123413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.113228    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-podinfo\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.123413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.113266    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-credential-socket\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.123413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.113303    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2xzs\" (UniqueName: \"kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-kube-api-access-d2xzs\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.123413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.113345    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-data\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.123413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.113399    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istiod-ca-cert\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.123413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.113427    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-envoy\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.214440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.214282    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-token\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.214440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.214346    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-podinfo\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.214440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.214430    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-credential-socket\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.214820 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.214484    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2xzs\" (UniqueName: \"kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-kube-api-access-d2xzs\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.214820 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.214530    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-data\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.214820 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.214562    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istiod-ca-cert\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.214820 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.214595    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-envoy\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.214820 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.214687    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-certs\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.214820 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.214722    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-socket\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.215167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.215123    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-socket\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.216351 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.216322    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-credential-socket\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.217571 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.217545    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istiod-ca-cert\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.217839 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.217798    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-certs\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.218020 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.217967    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-data\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.236508 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.220222    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-envoy\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.236508 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.223421    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-podinfo\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.236508 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.232969    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2xzs\" (UniqueName: \"kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-kube-api-access-d2xzs\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.237735 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.237692    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-token\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.315409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.288805    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:54.468942 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.468874    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm" event={"ID":"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120","Type":"ContainerStarted","Data":"4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829"}
Nov 04 07:34:54.469062 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.469041    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"
Nov 04 07:34:54.493152 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.493098    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm" podStartSLOduration=1.833071299 podStartE2EDuration="6.493081046s" totalImagesPullingTime="4.660009747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:54.491241749 +0000 UTC m=+1116.513457011" watchObservedRunningTime="2026-11-04 07:34:54.493081046 +0000 UTC m=+1116.515296308"
Nov 04 07:34:54.599238 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.599139    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"]
Nov 04 07:34:54.605873 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.605834    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:34:54.606050 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.605906    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:34:54.606050 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:54.605940    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:34:55.476006 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.475971    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv" event={"ID":"9661dd1c-40ae-420f-96cf-af9a01477333","Type":"ContainerStarted","Data":"bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62"}
Nov 04 07:34:55.476465 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.476037    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"
Nov 04 07:34:55.484494 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.484458    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh" event={"ID":"7244c03f-1118-4bab-92ec-34894a8c9c5f","Type":"ContainerStarted","Data":"6802f9ca4da9fdd0fe9e988f5593d92454c719b5057f43ba2be0162eb5b25c97"}
Nov 04 07:34:55.484646 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.484557    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"
Nov 04 07:34:55.486214 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.486170    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn" event={"ID":"41718dd1-af1a-4e5b-953a-f9734b3f3483","Type":"ContainerStarted","Data":"c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012"}
Nov 04 07:34:55.486334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.486245    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:34:55.488092 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.488065    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k" event={"ID":"a8b6a0dc-8600-4ab9-923f-3d680473f39f","Type":"ContainerStarted","Data":"65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2"}
Nov 04 07:34:55.488509 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.488114    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"
Nov 04 07:34:55.489698 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.489676    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b" event={"ID":"3477aa8d-a28d-4e96-8a17-1f177c1080ff","Type":"ContainerStarted","Data":"ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23"}
Nov 04 07:34:55.489798 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.489726    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"
Nov 04 07:34:55.491139 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.491119    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc" event={"ID":"efa49e6f-5463-4724-a2b6-51882cb8b317","Type":"ContainerStarted","Data":"ce8b8de9dd46d7e59a5fa9209aa66e1507e11a2a929e9d128607ba5ff5d7ebd8"}
Nov 04 07:34:55.491227 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.491165    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"
Nov 04 07:34:55.492662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.492640    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" event={"ID":"693fd0ce-1ebc-4173-9d22-b8ed367c1a26","Type":"ContainerStarted","Data":"098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2"}
Nov 04 07:34:55.492758 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.492681    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" event={"ID":"693fd0ce-1ebc-4173-9d22-b8ed367c1a26","Type":"ContainerStarted","Data":"444680534a88e2e43c3d330b903392348bde26a1ab1e6ef4bdb80358bcf2d75a"}
Nov 04 07:34:55.495831 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.495791    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv" podStartSLOduration=1.7548101539999998 podStartE2EDuration="4.49578013s" totalImagesPullingTime="2.740969976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:55.495557808 +0000 UTC m=+1117.517773070" watchObservedRunningTime="2026-11-04 07:34:55.49578013 +0000 UTC m=+1117.517995394"
Nov 04 07:34:55.497227 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.497202    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt" event={"ID":"9b8fb283-b77c-4758-9497-84ba24890da2","Type":"ContainerStarted","Data":"0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce"}
Nov 04 07:34:55.515517 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.515481    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh" podStartSLOduration=1.79575108 podStartE2EDuration="6.515469015s" totalImagesPullingTime="4.719717935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:55.51495332 +0000 UTC m=+1117.537168597" watchObservedRunningTime="2026-11-04 07:34:55.515469015 +0000 UTC m=+1117.537684275"
Nov 04 07:34:55.541982 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.541944    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc" podStartSLOduration=1.835924477 podStartE2EDuration="6.541930484s" totalImagesPullingTime="4.706006007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:55.539932975 +0000 UTC m=+1117.562148237" watchObservedRunningTime="2026-11-04 07:34:55.541930484 +0000 UTC m=+1117.564145743"
Nov 04 07:34:55.567801 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.567743    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k" podStartSLOduration=2.395673176 podStartE2EDuration="5.567729564s" totalImagesPullingTime="3.172056388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:55.565570099 +0000 UTC m=+1117.587785357" watchObservedRunningTime="2026-11-04 07:34:55.567729564 +0000 UTC m=+1117.589944824"
Nov 04 07:34:55.585461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.585410    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn" podStartSLOduration=1.714464504 podStartE2EDuration="6.585395669s" totalImagesPullingTime="4.870931165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:55.584787739 +0000 UTC m=+1117.607003001" watchObservedRunningTime="2026-11-04 07:34:55.585395669 +0000 UTC m=+1117.607611031"
Nov 04 07:34:55.619720 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.619619    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" podStartSLOduration=2.619603405 podStartE2EDuration="2.619603405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:55.618914063 +0000 UTC m=+1117.641129324" watchObservedRunningTime="2026-11-04 07:34:55.619603405 +0000 UTC m=+1117.641818670"
Nov 04 07:34:55.653455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.653407    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b" podStartSLOduration=1.806144046 podStartE2EDuration="4.653394086s" totalImagesPullingTime="2.84725004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:55.653299894 +0000 UTC m=+1117.675515154" watchObservedRunningTime="2026-11-04 07:34:55.653394086 +0000 UTC m=+1117.675609344"
Nov 04 07:34:55.673144 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:55.673094    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt" podStartSLOduration=2.380742356 podStartE2EDuration="5.673077677s" totalImagesPullingTime="3.292335321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:55.671842478 +0000 UTC m=+1117.694057735" watchObservedRunningTime="2026-11-04 07:34:55.673077677 +0000 UTC m=+1117.695292937"
Nov 04 07:34:56.289526 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:56.289490    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:56.294192 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:56.294165    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:56.503954 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:56.503924    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:56.504382 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:56.503981    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"
Nov 04 07:34:56.504382 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:56.504032    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:34:59.499663 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.499630    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"]
Nov 04 07:34:59.504099 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.504082    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"
Nov 04 07:34:59.508578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.508550    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-29d3ee\""
Nov 04 07:34:59.513872 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.513852    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-29d3ee-dockercfg-ngzdt\""
Nov 04 07:34:59.519149 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.519121    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"]
Nov 04 07:34:59.522990 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.522967    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"]
Nov 04 07:34:59.523090 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.523079    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"
Nov 04 07:34:59.537088 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.537064    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"]
Nov 04 07:34:59.584342 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.584310    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xvlc\" (UniqueName: \"kubernetes.io/projected/e051d9eb-49d3-4277-8e05-a1ca78585d9f-kube-api-access-5xvlc\") pod \"payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg\" (UID: \"e051d9eb-49d3-4277-8e05-a1ca78585d9f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"
Nov 04 07:34:59.584541 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.584355    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snfgf\" (UniqueName: \"kubernetes.io/projected/6749a48e-433c-4073-9d11-1391830d9514-kube-api-access-snfgf\") pod \"payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5\" (UID: \"6749a48e-433c-4073-9d11-1391830d9514\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"
Nov 04 07:34:59.584541 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.584519    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6749a48e-433c-4073-9d11-1391830d9514-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5\" (UID: \"6749a48e-433c-4073-9d11-1391830d9514\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"
Nov 04 07:34:59.584639 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.584624    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e051d9eb-49d3-4277-8e05-a1ca78585d9f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg\" (UID: \"e051d9eb-49d3-4277-8e05-a1ca78585d9f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"
Nov 04 07:34:59.685755 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.685716    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6749a48e-433c-4073-9d11-1391830d9514-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5\" (UID: \"6749a48e-433c-4073-9d11-1391830d9514\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"
Nov 04 07:34:59.685945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.685832    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e051d9eb-49d3-4277-8e05-a1ca78585d9f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg\" (UID: \"e051d9eb-49d3-4277-8e05-a1ca78585d9f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"
Nov 04 07:34:59.685945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.685886    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xvlc\" (UniqueName: \"kubernetes.io/projected/e051d9eb-49d3-4277-8e05-a1ca78585d9f-kube-api-access-5xvlc\") pod \"payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg\" (UID: \"e051d9eb-49d3-4277-8e05-a1ca78585d9f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"
Nov 04 07:34:59.685945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.685929    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snfgf\" (UniqueName: \"kubernetes.io/projected/6749a48e-433c-4073-9d11-1391830d9514-kube-api-access-snfgf\") pod \"payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5\" (UID: \"6749a48e-433c-4073-9d11-1391830d9514\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"
Nov 04 07:34:59.686486 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.686462    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6749a48e-433c-4073-9d11-1391830d9514-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5\" (UID: \"6749a48e-433c-4073-9d11-1391830d9514\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"
Nov 04 07:34:59.686747 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.686727    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e051d9eb-49d3-4277-8e05-a1ca78585d9f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg\" (UID: \"e051d9eb-49d3-4277-8e05-a1ca78585d9f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"
Nov 04 07:34:59.698655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.698628    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xvlc\" (UniqueName: \"kubernetes.io/projected/e051d9eb-49d3-4277-8e05-a1ca78585d9f-kube-api-access-5xvlc\") pod \"payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg\" (UID: \"e051d9eb-49d3-4277-8e05-a1ca78585d9f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"
Nov 04 07:34:59.698753 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.698675    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snfgf\" (UniqueName: \"kubernetes.io/projected/6749a48e-433c-4073-9d11-1391830d9514-kube-api-access-snfgf\") pod \"payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5\" (UID: \"6749a48e-433c-4073-9d11-1391830d9514\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"
Nov 04 07:34:59.817744 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.817708    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"
Nov 04 07:34:59.837741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:34:59.837710    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"
Nov 04 07:35:00.100719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:00.100691    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"]
Nov 04 07:35:00.102223 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:35:00.102193    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode051d9eb_49d3_4277_8e05_a1ca78585d9f.slice/crio-d376ddf69e790a8e78acec66cc61446620d200b36554d73497111d625be43223 WatchSource:0}: Error finding container d376ddf69e790a8e78acec66cc61446620d200b36554d73497111d625be43223: Status 404 returned error can't find the container with id d376ddf69e790a8e78acec66cc61446620d200b36554d73497111d625be43223
Nov 04 07:35:00.127642 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:00.127551    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"]
Nov 04 07:35:00.132649 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:35:00.132620    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6749a48e_433c_4073_9d11_1391830d9514.slice/crio-a9b59029ab00865c7e222368675d8baa00602d30a7cee70329e157e56f7ec268 WatchSource:0}: Error finding container a9b59029ab00865c7e222368675d8baa00602d30a7cee70329e157e56f7ec268: Status 404 returned error can't find the container with id a9b59029ab00865c7e222368675d8baa00602d30a7cee70329e157e56f7ec268
Nov 04 07:35:00.528669 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:00.528622    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5" event={"ID":"6749a48e-433c-4073-9d11-1391830d9514","Type":"ContainerStarted","Data":"f868fc4aaba50f8f85695c21b9775966d3c873f1b86893a4ad811de1b4d7ba0d"}
Nov 04 07:35:00.529135 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:00.528681    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5" event={"ID":"6749a48e-433c-4073-9d11-1391830d9514","Type":"ContainerStarted","Data":"a9b59029ab00865c7e222368675d8baa00602d30a7cee70329e157e56f7ec268"}
Nov 04 07:35:00.529135 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:00.528710    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"
Nov 04 07:35:00.530087 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:00.530060    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg" event={"ID":"e051d9eb-49d3-4277-8e05-a1ca78585d9f","Type":"ContainerStarted","Data":"dfac28bec78408d1dc88e407766a855d6b2f7e4c72a4bc8292a85dfd75926e8b"}
Nov 04 07:35:00.530230 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:00.530105    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg" event={"ID":"e051d9eb-49d3-4277-8e05-a1ca78585d9f","Type":"ContainerStarted","Data":"d376ddf69e790a8e78acec66cc61446620d200b36554d73497111d625be43223"}
Nov 04 07:35:00.530230 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:00.530134    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"
Nov 04 07:35:00.549423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:00.549352    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5" podStartSLOduration=1.5493395250000002 podStartE2EDuration="1.549339525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:35:00.546487525 +0000 UTC m=+1122.568702783" watchObservedRunningTime="2026-11-04 07:35:00.549339525 +0000 UTC m=+1122.571554784"
Nov 04 07:35:00.568248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:00.568197    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg" podStartSLOduration=1.5681833699999999 podStartE2EDuration="1.56818337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:35:00.566780943 +0000 UTC m=+1122.588996204" watchObservedRunningTime="2026-11-04 07:35:00.56818337 +0000 UTC m=+1122.590398630"
Nov 04 07:35:05.501063 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:05.501035    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"
Nov 04 07:35:06.176915 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:35:06.176870    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:35:06.506398 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:06.506286    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"
Nov 04 07:35:06.506781 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:06.506490    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"
Nov 04 07:35:06.506781 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:06.506558    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"
Nov 04 07:35:06.506781 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:06.506636    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"
Nov 04 07:35:06.506781 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:06.506681    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"
Nov 04 07:35:06.506781 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:06.506736    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:35:07.512743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:07.512708    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"
Nov 04 07:35:10.707225 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.707192    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"]
Nov 04 07:35:10.710922 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.710898    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.720060 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.720040    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-1f6ce4\"/\"openshift-service-ca.crt\""
Nov 04 07:35:10.720180 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.720111    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-1f6ce4\"/\"kube-root-ca.crt\""
Nov 04 07:35:10.733799 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.733780    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"]
Nov 04 07:35:10.783920 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.782030    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-1f6ce4\"/\"simulator-w4-1f6ce4-kserve-self-signed-certs\""
Nov 04 07:35:10.783920 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.782162    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-1f6ce4\"/\"default-dockercfg-mssrq\""
Nov 04 07:35:10.801947 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.801918    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-dshm\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.802095 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.801962    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tls-certs\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.802095 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.802069    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tmp-dir\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.802213 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.802197    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74s2f\" (UniqueName: \"kubernetes.io/projected/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-kube-api-access-74s2f\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.802258 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.802228    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-model-cache\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.802330 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.802315    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-home\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.903668 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.903634    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-home\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.903668 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.903674    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-dshm\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.903979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.903689    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tls-certs\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.904192 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.903827    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tmp-dir\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.904192 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.904174    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-home\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.904447 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.904229    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tmp-dir\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.904447 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.904430    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74s2f\" (UniqueName: \"kubernetes.io/projected/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-kube-api-access-74s2f\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.904555 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.904471    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-model-cache\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.904876 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.904826    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-model-cache\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.906456 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.906431    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-dshm\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.906850 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.906831    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tls-certs\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:10.918331 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:10.918304    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74s2f\" (UniqueName: \"kubernetes.io/projected/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-kube-api-access-74s2f\") pod \"simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") " pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:11.022824 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:11.022710    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:11.332500 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:11.332478    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"]
Nov 04 07:35:11.334608 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:35:11.334589    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb915520d_c11d_402e_b0ff_c01dfd5cdc4f.slice/crio-7c726e4f192c3af3001110b58faa9b36954e7fae49a5fb45514d23534027c6b6 WatchSource:0}: Error finding container 7c726e4f192c3af3001110b58faa9b36954e7fae49a5fb45514d23534027c6b6: Status 404 returned error can't find the container with id 7c726e4f192c3af3001110b58faa9b36954e7fae49a5fb45514d23534027c6b6
Nov 04 07:35:11.562630 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:11.562591    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"
Nov 04 07:35:11.562806 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:11.562692    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"
Nov 04 07:35:11.592441 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:11.592331    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" event={"ID":"b915520d-c11d-402e-b0ff-c01dfd5cdc4f","Type":"ContainerStarted","Data":"a795dc778676f539d477b36173eb1d46bae773433bce65c70ca87771d467acf3"}
Nov 04 07:35:11.592441 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:11.592384    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" event={"ID":"b915520d-c11d-402e-b0ff-c01dfd5cdc4f","Type":"ContainerStarted","Data":"7c726e4f192c3af3001110b58faa9b36954e7fae49a5fb45514d23534027c6b6"}
Nov 04 07:35:11.667393 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:11.667320    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podStartSLOduration=1.667295346 podStartE2EDuration="1.667295346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:35:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:35:11.66672136 +0000 UTC m=+1133.688936621" watchObservedRunningTime="2026-11-04 07:35:11.667295346 +0000 UTC m=+1133.689510605"
Nov 04 07:35:12.022908 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:12.022808    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:16.045794 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.045749    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"]
Nov 04 07:35:16.049418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.049384    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.053340 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.053272    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-016e6c\"/\"kube-root-ca.crt\""
Nov 04 07:35:16.054495 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.054470    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-016e6c\"/\"openshift-service-ca.crt\""
Nov 04 07:35:16.054672 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.054550    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-016e6c\"/\"simulator-w2-016e6c-kserve-self-signed-certs\""
Nov 04 07:35:16.054672 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.054577    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-016e6c\"/\"default-dockercfg-dvhvh\""
Nov 04 07:35:16.060928 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.060882    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"]
Nov 04 07:35:16.171833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.171777    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g428\" (UniqueName: \"kubernetes.io/projected/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-kube-api-access-6g428\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.172013 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.171845    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tmp-dir\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.172013 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.171996    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-model-cache\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.172103 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.172048    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-home\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.172156 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.172118    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tls-certs\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.172265 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.172245    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-dshm\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.273701 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.273657    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-home\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.273701 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.273705    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tls-certs\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.273949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.273788    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-dshm\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.273949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.273838    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g428\" (UniqueName: \"kubernetes.io/projected/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-kube-api-access-6g428\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.273949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.273866    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tmp-dir\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.273949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.273935    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-model-cache\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.274167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.274098    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-home\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.274292 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.274264    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-model-cache\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.274521 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.274375    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tmp-dir\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.276952 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.276903    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-dshm\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.277135 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.277118    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tls-certs\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.286086 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.286059    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g428\" (UniqueName: \"kubernetes.io/projected/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-kube-api-access-6g428\") pod \"simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") " pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.364937 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.364847    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:16.647851 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:16.647811    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"]
Nov 04 07:35:16.650275 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:35:16.650242    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4b5d45c_0b31_4c2d_abd2_46c78c3a2880.slice/crio-df9dfdb970ecd87ab97f7464861e24e1468745a640f2db6ed886ac739774a1ca WatchSource:0}: Error finding container df9dfdb970ecd87ab97f7464861e24e1468745a640f2db6ed886ac739774a1ca: Status 404 returned error can't find the container with id df9dfdb970ecd87ab97f7464861e24e1468745a640f2db6ed886ac739774a1ca
Nov 04 07:35:17.632796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:17.632760    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" event={"ID":"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880","Type":"ContainerStarted","Data":"59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684"}
Nov 04 07:35:17.633292 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:17.632803    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" event={"ID":"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880","Type":"ContainerStarted","Data":"df9dfdb970ecd87ab97f7464861e24e1468745a640f2db6ed886ac739774a1ca"}
Nov 04 07:35:17.634498 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:17.634465    5626 generic.go:356] "Generic (PLEG): container finished" podID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerID="098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2" exitCode=0
Nov 04 07:35:17.634617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:17.634526    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" event={"ID":"693fd0ce-1ebc-4173-9d22-b8ed367c1a26","Type":"ContainerDied","Data":"098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2"}
Nov 04 07:35:17.634836 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:17.634824    5626 scope.go:122] "RemoveContainer" containerID="098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2"
Nov 04 07:35:17.636175 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:17.636137    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:35:17.636291 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:17.636215    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:35:17.636291 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:17.636283    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:35:17.656724 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:17.653486    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podStartSLOduration=1.6534719610000002 podStartE2EDuration="1.653471961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:35:17.653273218 +0000 UTC m=+1139.675488479" watchObservedRunningTime="2026-11-04 07:35:17.653471961 +0000 UTC m=+1139.675687223"
Nov 04 07:35:18.023994 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.023898    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"]
Nov 04 07:35:18.027792 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.027769    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.032375 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.032335    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-df3be8\"/\"kube-root-ca.crt\""
Nov 04 07:35:18.036603 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.036577    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-df3be8\"/\"openshift-service-ca.crt\""
Nov 04 07:35:18.037871 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.037700    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-df3be8\"/\"default-dockercfg-s79pk\""
Nov 04 07:35:18.037871 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.037843    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-df3be8\"/\"simulator-w0-df3be8-kserve-self-signed-certs\""
Nov 04 07:35:18.055428 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.052109    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"]
Nov 04 07:35:18.092102 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.092070    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzgft\" (UniqueName: \"kubernetes.io/projected/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-kube-api-access-mzgft\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.092310 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.092108    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tmp-dir\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.092310 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.092158    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-home\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.092310 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.092260    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-model-cache\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.092484 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.092331    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-dshm\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.092484 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.092458    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tls-certs\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.193792 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.193755    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-home\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.194049 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.193806    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-model-cache\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.194049 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.193880    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-dshm\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.194049 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.193937    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tls-certs\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.194049 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.194008    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzgft\" (UniqueName: \"kubernetes.io/projected/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-kube-api-access-mzgft\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.194049 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.194037    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tmp-dir\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.195047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.194882    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-home\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.196221 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.196195    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tmp-dir\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.196408 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.196303    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-model-cache\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.198393 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.198336    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-df3be8\"/\"simulator-w0-df3be8-kserve-self-signed-certs\""
Nov 04 07:35:18.199621 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.199593    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-dshm\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.208220 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.208190    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-df3be8\"/\"kube-root-ca.crt\""
Nov 04 07:35:18.208506 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.208267    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tls-certs\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.219280 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.219251    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-df3be8\"/\"openshift-service-ca.crt\""
Nov 04 07:35:18.229723 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.229690    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzgft\" (UniqueName: \"kubernetes.io/projected/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-kube-api-access-mzgft\") pod \"simulator-w0-df3be8-kserve-7d67d8c5b9-76j25\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") " pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.364265 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.364218    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-df3be8\"/\"default-dockercfg-s79pk\""
Nov 04 07:35:18.365230 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.365200    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:18.371504 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.371477    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:18.666944 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.666783    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" event={"ID":"693fd0ce-1ebc-4173-9d22-b8ed367c1a26","Type":"ContainerStarted","Data":"ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f"}
Nov 04 07:35:18.678763 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:18.678700    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"]
Nov 04 07:35:18.680338 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:35:18.680305    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24be67c2_869c_4e5b_ad7c_8bf19b2ee6cd.slice/crio-ff40c9b996d1c98fe8b72e151ddfca218a648c90cef3c4ac1c49139b9c16f6ce WatchSource:0}: Error finding container ff40c9b996d1c98fe8b72e151ddfca218a648c90cef3c4ac1c49139b9c16f6ce: Status 404 returned error can't find the container with id ff40c9b996d1c98fe8b72e151ddfca218a648c90cef3c4ac1c49139b9c16f6ce
Nov 04 07:35:19.289509 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:19.289476    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:35:19.289702 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:19.289661    5626 patch_prober.go:29] interesting pod/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 07:35:19.289749 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:19.289697    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 07:35:19.673830 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:19.673732    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" event={"ID":"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd","Type":"ContainerStarted","Data":"3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc"}
Nov 04 07:35:19.673830 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:19.673787    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" event={"ID":"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd","Type":"ContainerStarted","Data":"ff40c9b996d1c98fe8b72e151ddfca218a648c90cef3c4ac1c49139b9c16f6ce"}
Nov 04 07:35:19.710424 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:19.710345    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podStartSLOduration=1.710327361 podStartE2EDuration="1.710327361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:35:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:35:19.707574472 +0000 UTC m=+1141.729789732" watchObservedRunningTime="2026-11-04 07:35:19.710327361 +0000 UTC m=+1141.732542622"
Nov 04 07:35:20.289413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:20.289352    5626 patch_prober.go:29] interesting pod/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 07:35:20.289597 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:20.289430    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 07:35:20.372663 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:20.372614    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:21.023613 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:21.023581    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:21.037341 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:21.037313    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:21.289971 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:21.289930    5626 patch_prober.go:29] interesting pod/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 07:35:21.290184 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:21.290000    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 07:35:21.697356 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:21.697269    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:35:22.289560 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:22.289524    5626 patch_prober.go:29] interesting pod/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 07:35:22.289949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:22.289589    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 07:35:23.290295 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:23.290259    5626 patch_prober.go:29] interesting pod/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 07:35:23.290681 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:23.290323    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 07:35:24.294117 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:24.294086    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:35:24.294565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:24.294477    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:35:24.295159 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:24.295139    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:35:25.012880 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.012842    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"]
Nov 04 07:35:25.019224 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.019195    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.029648 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.029578    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-29d3ee\"/\"openshift-service-ca.crt\""
Nov 04 07:35:25.029648 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.029600    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-29d3ee\"/\"kube-root-ca.crt\""
Nov 04 07:35:25.029648 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.029596    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-29d3ee\"/\"simulator-w1-29d3ee-kserve-self-signed-certs\""
Nov 04 07:35:25.029909 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.029663    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-29d3ee\"/\"default-dockercfg-tts4m\""
Nov 04 07:35:25.030872 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.030850    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"]
Nov 04 07:35:25.103879 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.103834    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-model-cache\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.104091 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.103886    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tls-certs\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.104091 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.103978    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tmp-dir\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.104091 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.104029    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-dshm\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.104254 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.104123    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-home\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.104254 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.104173    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfslg\" (UniqueName: \"kubernetes.io/projected/44b06376-8c4a-45ac-a97e-a6e1cb982b32-kube-api-access-lfslg\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.204844 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.204799    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-model-cache\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.205120 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.205090    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tls-certs\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.205214 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.205150    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tmp-dir\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.205214 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.205156    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-model-cache\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.205338 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.205259    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-dshm\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.205338 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.205328    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-home\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.205634 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.205477    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfslg\" (UniqueName: \"kubernetes.io/projected/44b06376-8c4a-45ac-a97e-a6e1cb982b32-kube-api-access-lfslg\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.205634 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.205494    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tmp-dir\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.205744 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.205660    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-home\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.207547 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.207525    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-dshm\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.207700 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.207681    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tls-certs\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.218421 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.218354    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfslg\" (UniqueName: \"kubernetes.io/projected/44b06376-8c4a-45ac-a97e-a6e1cb982b32-kube-api-access-lfslg\") pod \"simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") " pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.334901 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.334865    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:25.614851 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.614823    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"]
Nov 04 07:35:25.618016 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:35:25.617984    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44b06376_8c4a_45ac_a97e_a6e1cb982b32.slice/crio-692783ffbbab57b4a4256d37f06a9ea67169439f57571079b54b9c136ac55bf5 WatchSource:0}: Error finding container 692783ffbbab57b4a4256d37f06a9ea67169439f57571079b54b9c136ac55bf5: Status 404 returned error can't find the container with id 692783ffbbab57b4a4256d37f06a9ea67169439f57571079b54b9c136ac55bf5
Nov 04 07:35:25.716591 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.716555    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" event={"ID":"44b06376-8c4a-45ac-a97e-a6e1cb982b32","Type":"ContainerStarted","Data":"64a3e15b7acb57060e1809d5b00fceb29f218d07846023226c2789d90d29a00d"}
Nov 04 07:35:25.716746 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.716604    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" event={"ID":"44b06376-8c4a-45ac-a97e-a6e1cb982b32","Type":"ContainerStarted","Data":"692783ffbbab57b4a4256d37f06a9ea67169439f57571079b54b9c136ac55bf5"}
Nov 04 07:35:25.740551 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:25.740484    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podStartSLOduration=1.740467481 podStartE2EDuration="1.740467481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:35:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:35:25.738348033 +0000 UTC m=+1147.760563293" watchObservedRunningTime="2026-11-04 07:35:25.740467481 +0000 UTC m=+1147.762682740"
Nov 04 07:35:26.335381 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:26.335323    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:26.365523 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:26.365488    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:26.383142 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:26.383101    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:26.737948 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:26.737851    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:35:28.372518 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:28.372479    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:28.392489 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:28.392459    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:28.750464 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:28.750386    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:35:32.537387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.537328    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"]
Nov 04 07:35:32.544271 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.544247    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.550671 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.550641    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-1f6ce4\"/\"premium-w4-1f6ce4-kserve-self-signed-certs\""
Nov 04 07:35:32.552861 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.552835    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"]
Nov 04 07:35:32.587768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.587732    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-model-cache\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.587949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.587783    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-dshm\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.587949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.587894    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-tmp-dir\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.587949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.587923    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-home\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.588074 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.588017    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt28t\" (UniqueName: \"kubernetes.io/projected/6629b726-b52a-4100-9bfd-ac8f4faca253-kube-api-access-pt28t\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.588074 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.588049    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6629b726-b52a-4100-9bfd-ac8f4faca253-tls-certs\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.689352 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.689288    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-tmp-dir\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.689352 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.689353    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-home\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.689694 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.689450    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt28t\" (UniqueName: \"kubernetes.io/projected/6629b726-b52a-4100-9bfd-ac8f4faca253-kube-api-access-pt28t\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.689694 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.689482    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6629b726-b52a-4100-9bfd-ac8f4faca253-tls-certs\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.689694 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.689682    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-model-cache\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.689938 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.689776    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-dshm\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.690984 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.690953    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-home\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.691259 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.691217    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-model-cache\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.692440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.692401    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-tmp-dir\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.693740 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.693686    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-dshm\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.694393 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.694346    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6629b726-b52a-4100-9bfd-ac8f4faca253-tls-certs\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.703794 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.703753    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt28t\" (UniqueName: \"kubernetes.io/projected/6629b726-b52a-4100-9bfd-ac8f4faca253-kube-api-access-pt28t\") pod \"premium-w4-1f6ce4-kserve-5db4676b8f-hfks6\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") " pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:32.861978 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:32.861888    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:33.145242 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:33.145217    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"]
Nov 04 07:35:33.146917 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:35:33.146892    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6629b726_b52a_4100_9bfd_ac8f4faca253.slice/crio-f3f7e882db42e7ff847e1211e64d8fbb07d7c5747b265eaec5821774e186fb13 WatchSource:0}: Error finding container f3f7e882db42e7ff847e1211e64d8fbb07d7c5747b265eaec5821774e186fb13: Status 404 returned error can't find the container with id f3f7e882db42e7ff847e1211e64d8fbb07d7c5747b265eaec5821774e186fb13
Nov 04 07:35:33.767733 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:33.767698    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" event={"ID":"6629b726-b52a-4100-9bfd-ac8f4faca253","Type":"ContainerStarted","Data":"0702c6a1e941f4aa7bea140ebe7d8714eceb186904117e5c9b4fe6c1739f4a9b"}
Nov 04 07:35:33.768163 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:33.767743    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" event={"ID":"6629b726-b52a-4100-9bfd-ac8f4faca253","Type":"ContainerStarted","Data":"f3f7e882db42e7ff847e1211e64d8fbb07d7c5747b265eaec5821774e186fb13"}
Nov 04 07:35:33.788962 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:33.788910    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podStartSLOduration=1.788891822 podStartE2EDuration="1.788891822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:35:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:35:33.788069127 +0000 UTC m=+1155.810284388" watchObservedRunningTime="2026-11-04 07:35:33.788891822 +0000 UTC m=+1155.811107083"
Nov 04 07:35:33.862886 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:33.862861    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:35.335679 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:35.335647    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:35.348446 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:35.348423    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:35.789832 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:35.789797    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:35:36.172349 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:35:36.172260    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:35:42.862248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:42.862213    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:42.874983 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:42.874954    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:43.842340 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:43.842311    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:35:44.123004 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.122932    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"]
Nov 04 07:35:44.126693 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.126675    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.133577 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.133554    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-016e6c-openshift-default-dockercfg-d2ms9\""
Nov 04 07:35:44.140409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.140376    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"]
Nov 04 07:35:44.214108 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.214078    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.214108 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.214110    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrwqx\" (UniqueName: \"kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-kube-api-access-xrwqx\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.214322 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.214284    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.214322 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.214315    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.214433 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.214355    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.214490 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.214472    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.214534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.214512    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.214610 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.214594    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.214739 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.214724    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.315541 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.315503    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.315732 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.315613    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.315732 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.315696    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.315732 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.315719    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrwqx\" (UniqueName: \"kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-kube-api-access-xrwqx\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.315901 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.315872    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.315951 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.315908    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.316014 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.315991    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.316070 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.316035    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.316132 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.316072    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.316132 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.316104    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.316239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.316207    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.316394 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.316377    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.316462 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.316429    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.316844 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.316820    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.317983 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.317958    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.318517 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.318498    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.327997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.327950    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.327997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.327974    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrwqx\" (UniqueName: \"kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-kube-api-access-xrwqx\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.441280 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.441207    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:44.724104 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.724077    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"]
Nov 04 07:35:44.725956 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:35:44.725930    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfdec595_bd19_4d94_9a93_5bfc67f9de4e.slice/crio-90c7bd6a1e5f17612c2b0ede2f4d3d75c72efad3a4f7737d7fdc3fb3d832fced WatchSource:0}: Error finding container 90c7bd6a1e5f17612c2b0ede2f4d3d75c72efad3a4f7737d7fdc3fb3d832fced: Status 404 returned error can't find the container with id 90c7bd6a1e5f17612c2b0ede2f4d3d75c72efad3a4f7737d7fdc3fb3d832fced
Nov 04 07:35:44.728357 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.728322    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:35:44.728499 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.728435    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:35:44.728499 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.728482    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:35:44.838947 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.838905    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" event={"ID":"cfdec595-bd19-4d94-9a93-5bfc67f9de4e","Type":"ContainerStarted","Data":"9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2"}
Nov 04 07:35:44.839142 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.838973    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" event={"ID":"cfdec595-bd19-4d94-9a93-5bfc67f9de4e","Type":"ContainerStarted","Data":"90c7bd6a1e5f17612c2b0ede2f4d3d75c72efad3a4f7737d7fdc3fb3d832fced"}
Nov 04 07:35:44.871879 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:44.871813    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" podStartSLOduration=0.871799775 podStartE2EDuration="871.799775ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:35:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:35:44.86886454 +0000 UTC m=+1166.891079811" watchObservedRunningTime="2026-11-04 07:35:44.871799775 +0000 UTC m=+1166.894015034"
Nov 04 07:35:45.441510 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:45.441444    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:45.443212 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:45.443178    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 07:35:45.443381 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:45.443235    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" podUID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 07:35:46.460386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:46.459605    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 07:35:46.460386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:46.459674    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" podUID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 07:35:47.450386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:47.442657    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 07:35:47.450386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:47.442716    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" podUID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 07:35:48.444392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:48.442681    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 07:35:48.444392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:48.442756    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" podUID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 07:35:49.452387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:49.441766    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 07:35:49.452387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:49.441827    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" podUID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 07:35:50.442387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:50.441816    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 07:35:50.442387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:50.441901    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" podUID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 07:35:51.441733 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:51.441697    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 07:35:51.442108 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:51.441762    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" podUID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 07:35:52.447130 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:52.447094    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:52.447964 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:52.447944    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:35:52.448702 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:35:52.448679    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:36:01.265256 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.265172    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"]
Nov 04 07:36:01.271388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.271347    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.275838 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.275807    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-df3be8-openshift-default-dockercfg-xkvb9\""
Nov 04 07:36:01.282578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.282558    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"]
Nov 04 07:36:01.315811 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.315782    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdtgv\" (UniqueName: \"kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-kube-api-access-rdtgv\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.315942 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.315845    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.315942 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.315890    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/646c74b6-7721-410a-bccf-004404e9c41a-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.316040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.315970    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.316040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.316003    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.316111 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.316090    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.316168 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.316154    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.316208 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.316181    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/646c74b6-7721-410a-bccf-004404e9c41a-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.316208 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.316203    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.417731 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.417701    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/646c74b6-7721-410a-bccf-004404e9c41a-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.417881 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.417741    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.417881 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.417842    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdtgv\" (UniqueName: \"kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-kube-api-access-rdtgv\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.417976 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.417884    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.417976 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.417902    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/646c74b6-7721-410a-bccf-004404e9c41a-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.418080 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.418053    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.418125 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.418106    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.418527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.418194    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.418527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.418305    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.418527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.418459    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.418527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.418500    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.418848 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.418667    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/646c74b6-7721-410a-bccf-004404e9c41a-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.419118 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.419094    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.419206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.419165    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.420337 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.420314    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/646c74b6-7721-410a-bccf-004404e9c41a-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.420738 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.420720    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.430515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.430492    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.430599 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.430524    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdtgv\" (UniqueName: \"kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-kube-api-access-rdtgv\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.586527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.586489    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:01.880835 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.879457    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"]
Nov 04 07:36:01.888646 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.888601    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:36:01.888955 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.888909    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:36:01.889065 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.889035    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:36:01.964779 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:01.964754    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz" event={"ID":"646c74b6-7721-410a-bccf-004404e9c41a","Type":"ContainerStarted","Data":"443a626ced16a2264e0ad0d3c3388540340854784457475a6849afca061ebe23"}
Nov 04 07:36:02.981974 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:02.981928    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz" event={"ID":"646c74b6-7721-410a-bccf-004404e9c41a","Type":"ContainerStarted","Data":"5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857"}
Nov 04 07:36:03.017019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:03.016949    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz" podStartSLOduration=2.016929674 podStartE2EDuration="2.016929674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:36:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:36:03.012103056 +0000 UTC m=+1185.034318319" watchObservedRunningTime="2026-11-04 07:36:03.016929674 +0000 UTC m=+1185.039144936"
Nov 04 07:36:03.587212 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:03.587170    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:03.589077 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:03.589049    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 07:36:03.589242 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:03.589105    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz" podUID="646c74b6-7721-410a-bccf-004404e9c41a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 07:36:04.587492 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:04.587447    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 07:36:04.587914 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:04.587516    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz" podUID="646c74b6-7721-410a-bccf-004404e9c41a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 07:36:05.587253 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:05.587221    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 07:36:05.587455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:05.587275    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz" podUID="646c74b6-7721-410a-bccf-004404e9c41a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 07:36:06.188882 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:36:06.188835    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:36:06.587030 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:06.586990    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 07:36:06.587228 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:06.587044    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz" podUID="646c74b6-7721-410a-bccf-004404e9c41a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 07:36:07.587146 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:07.587102    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 07:36:07.587595 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:07.587163    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz" podUID="646c74b6-7721-410a-bccf-004404e9c41a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 07:36:08.687894 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:08.687716    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:08.688689 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:08.688664    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:08.689524 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:08.689506    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:36:29.045063 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.045021    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"]
Nov 04 07:36:29.050462 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.050439    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.058128 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.058107    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-60ed4a-openshift-default-dockercfg-dcrsd\""
Nov 04 07:36:29.065246 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.065220    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"]
Nov 04 07:36:29.087293 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.087249    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.087293 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.087288    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.087573 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.087307    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.087573 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.087404    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.087698 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.087661    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.087797 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.087777    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdqjk\" (UniqueName: \"kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-kube-api-access-mdqjk\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.087878 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.087840    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.087935 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.087877    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.087935 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.087907    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.188601 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.188564    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.188780 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.188638    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdqjk\" (UniqueName: \"kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-kube-api-access-mdqjk\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.188780 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.188677    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.188780 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.188700    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.188780 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.188716    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.188780 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.188760    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.189017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.188785    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.189017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.188805    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.189017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.188840    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.189178 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.189143    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.189237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.189226    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.189293 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.189268    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.189293 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.189279    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.189491 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.189468    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.191255 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.191237    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.191385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.191347    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.201150 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.201129    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.201610 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.201589    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdqjk\" (UniqueName: \"kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-kube-api-access-mdqjk\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.364227 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.364144    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:29.648434 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.648406    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"]
Nov 04 07:36:29.650513 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:36:29.650480    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50ee65a2_eedf_47ac_93e3_0b75218a0d7b.slice/crio-a6c099970404c1b5aee644c217caf15b3e4e380388dc8a69e0c3f7febb3be2e7 WatchSource:0}: Error finding container a6c099970404c1b5aee644c217caf15b3e4e380388dc8a69e0c3f7febb3be2e7: Status 404 returned error can't find the container with id a6c099970404c1b5aee644c217caf15b3e4e380388dc8a69e0c3f7febb3be2e7
Nov 04 07:36:29.653042 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.653000    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:36:29.653135 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.653094    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:36:29.653178 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:29.653140    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:36:30.162498 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:30.162460    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl" event={"ID":"50ee65a2-eedf-47ac-93e3-0b75218a0d7b","Type":"ContainerStarted","Data":"f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358"}
Nov 04 07:36:30.162955 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:30.162547    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl" event={"ID":"50ee65a2-eedf-47ac-93e3-0b75218a0d7b","Type":"ContainerStarted","Data":"a6c099970404c1b5aee644c217caf15b3e4e380388dc8a69e0c3f7febb3be2e7"}
Nov 04 07:36:30.192087 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:30.192007    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl" podStartSLOduration=1.191981305 podStartE2EDuration="1.191981305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:36:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:36:30.187125987 +0000 UTC m=+1212.209341262" watchObservedRunningTime="2026-11-04 07:36:30.191981305 +0000 UTC m=+1212.214196566"
Nov 04 07:36:30.365284 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:30.365241    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:30.366810 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:30.366781    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 07:36:30.366867 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:30.366848    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl" podUID="50ee65a2-eedf-47ac-93e3-0b75218a0d7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 07:36:31.364979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:31.364936    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 07:36:31.365440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:31.365008    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl" podUID="50ee65a2-eedf-47ac-93e3-0b75218a0d7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 07:36:32.365429 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:32.365394    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 07:36:32.365867 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:32.365477    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl" podUID="50ee65a2-eedf-47ac-93e3-0b75218a0d7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 07:36:33.364892 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:33.364849    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 07:36:33.365076 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:33.364920    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl" podUID="50ee65a2-eedf-47ac-93e3-0b75218a0d7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 07:36:34.365172 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:34.365139    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 07:36:34.365662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:34.365194    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl" podUID="50ee65a2-eedf-47ac-93e3-0b75218a0d7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 07:36:35.367867 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:35.367792    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 07:36:35.368679 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:35.368636    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl" podUID="50ee65a2-eedf-47ac-93e3-0b75218a0d7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 07:36:36.173469 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:36:36.173424    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:36:36.472984 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:36.472723    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:36.475352 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:36.473962    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:36:36.476158 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:36:36.476130    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:37:01.126459 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.126429    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"]
Nov 04 07:37:01.131133 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.131110    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.143083 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.143061    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-60ed4a\"/\"openshift-service-ca.crt\""
Nov 04 07:37:01.149392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.149354    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-60ed4a\"/\"default-dockercfg-zs5mn\""
Nov 04 07:37:01.149526 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.149418    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-60ed4a\"/\"distinct-w3-60ed4a-kserve-self-signed-certs\""
Nov 04 07:37:01.150584 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.150561    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"]
Nov 04 07:37:01.201633 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.201596    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-home\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.201798 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.201662    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-model-cache\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.201845 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.201787    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-dshm\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.201880 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.201863    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7dz8\" (UniqueName: \"kubernetes.io/projected/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-kube-api-access-j7dz8\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.202004 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.201988    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tmp-dir\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.202040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.202022    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tls-certs\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.243208 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.243177    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-60ed4a\"/\"kube-root-ca.crt\""
Nov 04 07:37:01.303168 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.303130    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tmp-dir\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.303387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.303178    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tls-certs\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.303387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.303220    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-home\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.303387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.303271    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-model-cache\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.303387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.303316    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-dshm\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.303639 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.303405    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7dz8\" (UniqueName: \"kubernetes.io/projected/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-kube-api-access-j7dz8\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.304657 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.304598    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tmp-dir\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.304657 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.304635    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-model-cache\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.304870 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.304850    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-home\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.307229 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.307194    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tls-certs\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.308888 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.308863    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-dshm\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.316034 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.316014    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7dz8\" (UniqueName: \"kubernetes.io/projected/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-kube-api-access-j7dz8\") pod \"distinct-w3-60ed4a-kserve-dd89c878b-68l8x\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.445504 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.445421    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:01.726248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:01.726223    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"]
Nov 04 07:37:01.727813 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:37:01.727781    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefb9db9c_e485_4ad4_9ce8_e5fa791a11d7.slice/crio-7b615441d6ee8c485c5df68e33ca41b0250ef4fbdd49fe944045540eecf49d42 WatchSource:0}: Error finding container 7b615441d6ee8c485c5df68e33ca41b0250ef4fbdd49fe944045540eecf49d42: Status 404 returned error can't find the container with id 7b615441d6ee8c485c5df68e33ca41b0250ef4fbdd49fe944045540eecf49d42
Nov 04 07:37:02.389647 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:02.389613    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" event={"ID":"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7","Type":"ContainerStarted","Data":"15d59189554b3b3b0439abde9e9cef6fd6863c3f08e0fe4523db58985296b7d9"}
Nov 04 07:37:02.389647 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:02.389653    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" event={"ID":"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7","Type":"ContainerStarted","Data":"7b615441d6ee8c485c5df68e33ca41b0250ef4fbdd49fe944045540eecf49d42"}
Nov 04 07:37:02.420994 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:02.420944    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podStartSLOduration=1.420931949 podStartE2EDuration="1.420931949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:02.417967238 +0000 UTC m=+1244.440182499" watchObservedRunningTime="2026-11-04 07:37:02.420931949 +0000 UTC m=+1244.443147208"
Nov 04 07:37:02.446426 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:02.446392    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:03.494314 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:03.494278    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"]
Nov 04 07:37:03.498248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:03.498230    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"
Nov 04 07:37:03.509066 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:03.509039    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"]
Nov 04 07:37:03.562690 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:03.562635    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-6dd194\""
Nov 04 07:37:03.563161 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:03.563138    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-6dd194-dockercfg-k44tj\""
Nov 04 07:37:03.635061 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:03.635029    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9s9g\" (UniqueName: \"kubernetes.io/projected/a4c782a9-861e-4733-9c6e-41a55b0610b4-kube-api-access-p9s9g\") pod \"payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w\" (UID: \"a4c782a9-861e-4733-9c6e-41a55b0610b4\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"
Nov 04 07:37:03.635241 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:03.635088    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4c782a9-861e-4733-9c6e-41a55b0610b4-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w\" (UID: \"a4c782a9-861e-4733-9c6e-41a55b0610b4\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"
Nov 04 07:37:03.736521 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:03.736405    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4c782a9-861e-4733-9c6e-41a55b0610b4-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w\" (UID: \"a4c782a9-861e-4733-9c6e-41a55b0610b4\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"
Nov 04 07:37:03.737659 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:03.737346    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9s9g\" (UniqueName: \"kubernetes.io/projected/a4c782a9-861e-4733-9c6e-41a55b0610b4-kube-api-access-p9s9g\") pod \"payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w\" (UID: \"a4c782a9-861e-4733-9c6e-41a55b0610b4\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"
Nov 04 07:37:03.737659 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:03.737457    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4c782a9-861e-4733-9c6e-41a55b0610b4-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w\" (UID: \"a4c782a9-861e-4733-9c6e-41a55b0610b4\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"
Nov 04 07:37:03.749777 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:03.749718    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9s9g\" (UniqueName: \"kubernetes.io/projected/a4c782a9-861e-4733-9c6e-41a55b0610b4-kube-api-access-p9s9g\") pod \"payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w\" (UID: \"a4c782a9-861e-4733-9c6e-41a55b0610b4\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"
Nov 04 07:37:03.816347 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:03.816314    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"
Nov 04 07:37:04.116823 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:04.116791    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"]
Nov 04 07:37:04.118172 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:37:04.118147    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4c782a9_861e_4733_9c6e_41a55b0610b4.slice/crio-4e8f00d033991f934f81c091e4fbe035eb0bdf3dcb210a4fe12491706e2e81f3 WatchSource:0}: Error finding container 4e8f00d033991f934f81c091e4fbe035eb0bdf3dcb210a4fe12491706e2e81f3: Status 404 returned error can't find the container with id 4e8f00d033991f934f81c091e4fbe035eb0bdf3dcb210a4fe12491706e2e81f3
Nov 04 07:37:04.405675 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:04.405582    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w" event={"ID":"a4c782a9-861e-4733-9c6e-41a55b0610b4","Type":"ContainerStarted","Data":"d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848"}
Nov 04 07:37:04.405882 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:04.405869    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"
Nov 04 07:37:04.405926 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:04.405896    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w" event={"ID":"a4c782a9-861e-4733-9c6e-41a55b0610b4","Type":"ContainerStarted","Data":"4e8f00d033991f934f81c091e4fbe035eb0bdf3dcb210a4fe12491706e2e81f3"}
Nov 04 07:37:04.427658 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:04.427608    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w" podStartSLOduration=1.427592483 podStartE2EDuration="1.427592483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:04.424204756 +0000 UTC m=+1246.446420016" watchObservedRunningTime="2026-11-04 07:37:04.427592483 +0000 UTC m=+1246.449807742"
Nov 04 07:37:06.173281 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:37:06.173244    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:37:11.445587 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:11.445547    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:11.458760 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:11.458732    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:12.457978 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:12.457946    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:37:14.433852 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.433815    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"]
Nov 04 07:37:14.438782 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.438756    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.445912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.445878    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"]
Nov 04 07:37:14.446045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.445894    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-df3be8\"/\"distinct-2-w0-df3be8-kserve-self-signed-certs\""
Nov 04 07:37:14.502162 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.502127    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-model-cache\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.502162 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.502165    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf8055f2-ea85-4b51-a7ce-766f09b12126-tls-certs\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.502457 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.502242    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-tmp-dir\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.502498 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.502471    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-dshm\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.502601 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.502587    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-home\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.502642 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.502612    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg2xb\" (UniqueName: \"kubernetes.io/projected/cf8055f2-ea85-4b51-a7ce-766f09b12126-kube-api-access-qg2xb\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.603990 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.603955    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-home\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.604179 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.603996    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg2xb\" (UniqueName: \"kubernetes.io/projected/cf8055f2-ea85-4b51-a7ce-766f09b12126-kube-api-access-qg2xb\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.604179 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.604049    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-model-cache\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.604179 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.604067    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf8055f2-ea85-4b51-a7ce-766f09b12126-tls-certs\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.604179 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.604114    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-tmp-dir\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.604507 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.604187    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-dshm\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.604507 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.604444    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-home\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.604507 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.604492    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-model-cache\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.604661 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.604553    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-tmp-dir\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.606544 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.606518    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-dshm\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.606821 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.606803    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf8055f2-ea85-4b51-a7ce-766f09b12126-tls-certs\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.618932 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.618903    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg2xb\" (UniqueName: \"kubernetes.io/projected/cf8055f2-ea85-4b51-a7ce-766f09b12126-kube-api-access-qg2xb\") pod \"distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") " pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:14.754737 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:14.754639    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:15.035350 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:15.035324    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"]
Nov 04 07:37:15.036964 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:37:15.036938    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf8055f2_ea85_4b51_a7ce_766f09b12126.slice/crio-38befc5c795fed62b5fb8037b5028713cf23b69c27f6fcd21868f5622ec945c3 WatchSource:0}: Error finding container 38befc5c795fed62b5fb8037b5028713cf23b69c27f6fcd21868f5622ec945c3: Status 404 returned error can't find the container with id 38befc5c795fed62b5fb8037b5028713cf23b69c27f6fcd21868f5622ec945c3
Nov 04 07:37:15.416032 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:15.415951    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"
Nov 04 07:37:15.479617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:15.479579    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" event={"ID":"cf8055f2-ea85-4b51-a7ce-766f09b12126","Type":"ContainerStarted","Data":"f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097"}
Nov 04 07:37:15.480027 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:15.479634    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" event={"ID":"cf8055f2-ea85-4b51-a7ce-766f09b12126","Type":"ContainerStarted","Data":"38befc5c795fed62b5fb8037b5028713cf23b69c27f6fcd21868f5622ec945c3"}
Nov 04 07:37:15.501460 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:15.501400    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podStartSLOduration=1.501384783 podStartE2EDuration="1.501384783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:15.499674315 +0000 UTC m=+1257.521889574" watchObservedRunningTime="2026-11-04 07:37:15.501384783 +0000 UTC m=+1257.523600065"
Nov 04 07:37:15.754847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:15.754763    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:16.390904 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.390871    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"]
Nov 04 07:37:16.403231 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.403202    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.407989 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.407956    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"]
Nov 04 07:37:16.430464 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.430433    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.430638 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.430479    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.430638 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.430503    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.430762 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.430682    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.430802 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.430782    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6nhg\" (UniqueName: \"kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-kube-api-access-m6nhg\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.430849 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.430836    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.430894 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.430866    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.430936 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.430895    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.431010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.430989    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.531899 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.531859    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532321 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.531924    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6nhg\" (UniqueName: \"kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-kube-api-access-m6nhg\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532321 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.531981    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532321 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.532117    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532321 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.532167    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532321 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.532224    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532321 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.532302    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532546 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.532407    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532546 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.532454    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.532709    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532890 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.532819    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532890 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.532857    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532990 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.532922    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.532990 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.532983    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.534230 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.534209    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.534831 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.534810    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.544491 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.544461    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.544865 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.544841    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6nhg\" (UniqueName: \"kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-kube-api-access-m6nhg\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:16.722716 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:16.722622    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:17.007015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:17.006984    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"]
Nov 04 07:37:17.007975 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:37:17.007944    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb5f0d1c_bf2b_4f0d_8556_a85713f541aa.slice/crio-eee25e3cd0b8c80f7b2b909c2256d6e74a28c009b080916d412568a59117b698 WatchSource:0}: Error finding container eee25e3cd0b8c80f7b2b909c2256d6e74a28c009b080916d412568a59117b698: Status 404 returned error can't find the container with id eee25e3cd0b8c80f7b2b909c2256d6e74a28c009b080916d412568a59117b698
Nov 04 07:37:17.010696 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:17.010657    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:17.010785 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:17.010747    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:17.010821 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:17.010795    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:17.494236 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:17.494199    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" event={"ID":"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa","Type":"ContainerStarted","Data":"fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44"}
Nov 04 07:37:17.494458 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:17.494264    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" event={"ID":"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa","Type":"ContainerStarted","Data":"eee25e3cd0b8c80f7b2b909c2256d6e74a28c009b080916d412568a59117b698"}
Nov 04 07:37:17.539656 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:17.539575    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" podStartSLOduration=1.539553922 podStartE2EDuration="1.539553922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:17.539079708 +0000 UTC m=+1259.561294967" watchObservedRunningTime="2026-11-04 07:37:17.539553922 +0000 UTC m=+1259.561769183"
Nov 04 07:37:17.723187 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:17.723155    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:18.724555 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:18.724514    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 07:37:18.724939 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:18.724564    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 07:37:19.723475 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:19.723435    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 07:37:19.723685 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:19.723506    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 07:37:20.724025 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:20.723982    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 07:37:20.724550 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:20.724064    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 07:37:21.723308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:21.723264    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 07:37:21.723527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:21.723353    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 07:37:22.723844 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:22.723796    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 07:37:22.724219 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:22.723870    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 07:37:23.723222 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:23.723190    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 07:37:23.723426 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:23.723255    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 07:37:24.755747 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:24.755708    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:24.768593 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:24.768564    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:25.564957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:25.564920    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:37:25.764496 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:25.764454    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:37:25.765001 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:25.764500    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:37:25.878771 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:25.878674    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:25.879172 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:25.879131    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:25.880615 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:25.880591    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:37:28.768046 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:28.768015    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"]
Nov 04 07:37:28.772647 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:28.772622    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"
Nov 04 07:37:28.784795 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:28.784751    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-812032-dockercfg-zsc76\""
Nov 04 07:37:28.788639 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:28.788616    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"]
Nov 04 07:37:28.838812 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:28.838712    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-812032\""
Nov 04 07:37:28.924513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:28.924455    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzggd\" (UniqueName: \"kubernetes.io/projected/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-kube-api-access-qzggd\") pod \"payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp\" (UID: \"daaf0ef4-096a-4b48-9903-dabc9ec22ab9\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"
Nov 04 07:37:28.924680 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:28.924581    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp\" (UID: \"daaf0ef4-096a-4b48-9903-dabc9ec22ab9\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"
Nov 04 07:37:29.025586 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.025502    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzggd\" (UniqueName: \"kubernetes.io/projected/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-kube-api-access-qzggd\") pod \"payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp\" (UID: \"daaf0ef4-096a-4b48-9903-dabc9ec22ab9\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"
Nov 04 07:37:29.025586 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.025571    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp\" (UID: \"daaf0ef4-096a-4b48-9903-dabc9ec22ab9\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"
Nov 04 07:37:29.026297 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.026269    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp\" (UID: \"daaf0ef4-096a-4b48-9903-dabc9ec22ab9\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"
Nov 04 07:37:29.037996 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.037975    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzggd\" (UniqueName: \"kubernetes.io/projected/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-kube-api-access-qzggd\") pod \"payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp\" (UID: \"daaf0ef4-096a-4b48-9903-dabc9ec22ab9\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"
Nov 04 07:37:29.092521 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.092498    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"
Nov 04 07:37:29.285419 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.285385    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"]
Nov 04 07:37:29.291467 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.291443    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.312224 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.312192    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"]
Nov 04 07:37:29.381480 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.381447    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"]
Nov 04 07:37:29.382999 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:37:29.382971    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaaf0ef4_096a_4b48_9903_dabc9ec22ab9.slice/crio-b498a3e04a4bd2ea173aea596a4ff5f57b8b24e42f1c5049fbfb7857949540c9 WatchSource:0}: Error finding container b498a3e04a4bd2ea173aea596a4ff5f57b8b24e42f1c5049fbfb7857949540c9: Status 404 returned error can't find the container with id b498a3e04a4bd2ea173aea596a4ff5f57b8b24e42f1c5049fbfb7857949540c9
Nov 04 07:37:29.430345 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.430319    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.430470 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.430395    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.430558 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.430531    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.430613 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.430598    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.430769 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.430690    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.430817 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.430796    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.430862 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.430841    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.431768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.430937    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dcf696e-7927-43ac-a18b-f7e5128ca367-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.431768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.430988    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdplg\" (UniqueName: \"kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-kube-api-access-fdplg\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532037    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532082    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532107    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dcf696e-7927-43ac-a18b-f7e5128ca367-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532125    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdplg\" (UniqueName: \"kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-kube-api-access-fdplg\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532179    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532223    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532540 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532280    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532540 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532319    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532540 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532356    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532540 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532492    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532715 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532581    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532715 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532674    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532781 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532756    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.532883 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.532859    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dcf696e-7927-43ac-a18b-f7e5128ca367-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.534617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.534593    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.534751 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.534733    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.544825 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.544766    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.544903 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.544881    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdplg\" (UniqueName: \"kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-kube-api-access-fdplg\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.580722 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.580694    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp" event={"ID":"daaf0ef4-096a-4b48-9903-dabc9ec22ab9","Type":"ContainerStarted","Data":"f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c"}
Nov 04 07:37:29.580851 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.580740    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp" event={"ID":"daaf0ef4-096a-4b48-9903-dabc9ec22ab9","Type":"ContainerStarted","Data":"b498a3e04a4bd2ea173aea596a4ff5f57b8b24e42f1c5049fbfb7857949540c9"}
Nov 04 07:37:29.580851 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.580839    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"
Nov 04 07:37:29.604134 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.604083    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp" podStartSLOduration=1.604070474 podStartE2EDuration="1.604070474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:29.601496714 +0000 UTC m=+1271.623711974" watchObservedRunningTime="2026-11-04 07:37:29.604070474 +0000 UTC m=+1271.626285733"
Nov 04 07:37:29.605801 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.605779    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:29.886845 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.886815    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"]
Nov 04 07:37:29.888527 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:37:29.888497    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dcf696e_7927_43ac_a18b_f7e5128ca367.slice/crio-8f4a22f18554c66b6575d76fce38a3c944a647d95189a71fcb560cec0f4b01c2 WatchSource:0}: Error finding container 8f4a22f18554c66b6575d76fce38a3c944a647d95189a71fcb560cec0f4b01c2: Status 404 returned error can't find the container with id 8f4a22f18554c66b6575d76fce38a3c944a647d95189a71fcb560cec0f4b01c2
Nov 04 07:37:29.890867 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.890833    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:29.898469 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.891428    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:29.898469 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:29.891565    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:30.589446 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:30.589410    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" event={"ID":"5dcf696e-7927-43ac-a18b-f7e5128ca367","Type":"ContainerStarted","Data":"7c5b4e247628e7609d7178f73c079dc425c672fa621786055e11a85018ae540b"}
Nov 04 07:37:30.589620 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:30.589469    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" event={"ID":"5dcf696e-7927-43ac-a18b-f7e5128ca367","Type":"ContainerStarted","Data":"8f4a22f18554c66b6575d76fce38a3c944a647d95189a71fcb560cec0f4b01c2"}
Nov 04 07:37:30.605925 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:30.605900    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:30.607472 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:30.607450    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 07:37:30.607559 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:30.607499    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" podUID="5dcf696e-7927-43ac-a18b-f7e5128ca367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 07:37:30.616445 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:30.616405    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" podStartSLOduration=1.6163934009999998 podStartE2EDuration="1.616393401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:30.613702059 +0000 UTC m=+1272.635917318" watchObservedRunningTime="2026-11-04 07:37:30.616393401 +0000 UTC m=+1272.638608661"
Nov 04 07:37:31.606174 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:31.606135    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 07:37:31.606643 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:31.606189    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" podUID="5dcf696e-7927-43ac-a18b-f7e5128ca367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 07:37:32.606624 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:32.606593    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 07:37:32.606987 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:32.606639    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" podUID="5dcf696e-7927-43ac-a18b-f7e5128ca367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 07:37:33.606341 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:33.606304    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 07:37:33.606518 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:33.606371    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" podUID="5dcf696e-7927-43ac-a18b-f7e5128ca367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 07:37:34.606588 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:34.606540    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 07:37:34.606985 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:34.606611    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" podUID="5dcf696e-7927-43ac-a18b-f7e5128ca367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 07:37:35.097844 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.097812    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"]
Nov 04 07:37:35.102391 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.102346    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.115944 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.115923    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-2a8c48-openshift-default-dockercfg-x549v\""
Nov 04 07:37:35.120186 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.120164    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"]
Nov 04 07:37:35.197127 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.197095    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"]
Nov 04 07:37:35.203045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.203016    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.212965 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.212934    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-3da894-openshift-default-dockercfg-t4bfw\""
Nov 04 07:37:35.221735 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.221710    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fea08504-e401-4132-8b45-1f9b36646205-istio-podinfo\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.222579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.221781    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-istio-token\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.222579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.221821    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fea08504-e401-4132-8b45-1f9b36646205-istiod-ca-cert\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.222579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.221875    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g9cz\" (UniqueName: \"kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-kube-api-access-5g9cz\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.222579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.221930    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-data\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.222579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.221966    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-credential-socket\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.222579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.221989    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-certs\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.222579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.222092    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-envoy\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.222579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.222133    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-socket\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.223709 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.223679    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"]
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.322892    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-envoy\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.322933    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-data\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.322978    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-socket\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323010    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-envoy\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323051    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-socket\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323102    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fea08504-e401-4132-8b45-1f9b36646205-istio-podinfo\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323155    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-istio-token\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323218    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-istio-token\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323248    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-credential-socket\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323278    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-certs\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323565    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fea08504-e401-4132-8b45-1f9b36646205-istiod-ca-cert\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323648    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db38048a-323a-4ab4-a60a-a394546f78f6-istio-podinfo\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323720    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g9cz\" (UniqueName: \"kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-kube-api-access-5g9cz\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323752    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db38048a-323a-4ab4-a60a-a394546f78f6-istiod-ca-cert\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323841    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-data\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323907    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-credential-socket\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.323953    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-certs\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.324062    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftqwj\" (UniqueName: \"kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-kube-api-access-ftqwj\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.325317    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-socket\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.334685    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-data\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.335388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.335000    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-certs\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.371047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.346879    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-credential-socket\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.371047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.351948    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fea08504-e401-4132-8b45-1f9b36646205-istiod-ca-cert\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.371047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.355014    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fea08504-e401-4132-8b45-1f9b36646205-istio-podinfo\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.388538 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.388505    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g9cz\" (UniqueName: \"kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-kube-api-access-5g9cz\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.402410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.388980    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-envoy\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.402658 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.399875    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-istio-token\") pod \"e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") " pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.419292 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.419085    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:35.425223 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.425196    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-envoy\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.425343 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.425236    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-data\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.425343 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.425299    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-socket\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.425487 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.425470    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-istio-token\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.425575 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.425550    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-credential-socket\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.425642 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.425601    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-certs\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.425702 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.425683    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db38048a-323a-4ab4-a60a-a394546f78f6-istio-podinfo\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.425774 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.425754    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db38048a-323a-4ab4-a60a-a394546f78f6-istiod-ca-cert\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.425922 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.425881    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftqwj\" (UniqueName: \"kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-kube-api-access-ftqwj\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.426611 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.426235    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-data\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.426611 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.426566    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-credential-socket\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.426782 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.426572    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-certs\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.426958 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.426937    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-socket\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.427133 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.427110    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db38048a-323a-4ab4-a60a-a394546f78f6-istiod-ca-cert\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.429248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.429215    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db38048a-323a-4ab4-a60a-a394546f78f6-istio-podinfo\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.432981 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.432959    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-envoy\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.463069 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.463008    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftqwj\" (UniqueName: \"kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-kube-api-access-ftqwj\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.463919 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.463872    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-istio-token\") pod \"e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") " pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.520783 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.520754    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:35.607010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.606624    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 07:37:35.607010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.606681    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" podUID="5dcf696e-7927-43ac-a18b-f7e5128ca367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 07:37:35.763442 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.763407    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"]
Nov 04 07:37:35.767895 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:37:35.767858    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea08504_e401_4132_8b45_1f9b36646205.slice/crio-95aa391314356df51ade86e3e8bf97c2c8ee7be1ad4fa35486cf3afff72a70e6 WatchSource:0}: Error finding container 95aa391314356df51ade86e3e8bf97c2c8ee7be1ad4fa35486cf3afff72a70e6: Status 404 returned error can't find the container with id 95aa391314356df51ade86e3e8bf97c2c8ee7be1ad4fa35486cf3afff72a70e6
Nov 04 07:37:35.770571 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.770537    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:35.770637 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.770614    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:35.770685 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.770660    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:35.847844 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.847804    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"]
Nov 04 07:37:35.848640 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:37:35.848612    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb38048a_323a_4ab4_a60a_a394546f78f6.slice/crio-4b0f006cb725801ebf68f02f189619c772dfff876453f36d8d13c82410b80f8a WatchSource:0}: Error finding container 4b0f006cb725801ebf68f02f189619c772dfff876453f36d8d13c82410b80f8a: Status 404 returned error can't find the container with id 4b0f006cb725801ebf68f02f189619c772dfff876453f36d8d13c82410b80f8a
Nov 04 07:37:35.851959 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.851915    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:35.852044 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.852011    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:35.852096 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:35.852062    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:36.176758 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:37:36.176649    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:37:36.606697 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:36.606664    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 07:37:36.606886 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:36.606720    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" podUID="5dcf696e-7927-43ac-a18b-f7e5128ca367" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 07:37:36.632678 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:36.632636    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" event={"ID":"db38048a-323a-4ab4-a60a-a394546f78f6","Type":"ContainerStarted","Data":"bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b"}
Nov 04 07:37:36.633161 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:36.632698    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" event={"ID":"db38048a-323a-4ab4-a60a-a394546f78f6","Type":"ContainerStarted","Data":"4b0f006cb725801ebf68f02f189619c772dfff876453f36d8d13c82410b80f8a"}
Nov 04 07:37:36.634856 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:36.634824    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" event={"ID":"fea08504-e401-4132-8b45-1f9b36646205","Type":"ContainerStarted","Data":"1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea"}
Nov 04 07:37:36.635014 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:36.634881    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" event={"ID":"fea08504-e401-4132-8b45-1f9b36646205","Type":"ContainerStarted","Data":"95aa391314356df51ade86e3e8bf97c2c8ee7be1ad4fa35486cf3afff72a70e6"}
Nov 04 07:37:36.756149 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:36.756102    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" podStartSLOduration=1.75608539 podStartE2EDuration="1.75608539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:36.700695135 +0000 UTC m=+1278.722910396" watchObservedRunningTime="2026-11-04 07:37:36.75608539 +0000 UTC m=+1278.778300649"
Nov 04 07:37:36.757862 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:36.757826    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" podStartSLOduration=1.757819119 podStartE2EDuration="1.757819119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:36.754665874 +0000 UTC m=+1278.776881135" watchObservedRunningTime="2026-11-04 07:37:36.757819119 +0000 UTC m=+1278.780034379"
Nov 04 07:37:37.420241 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:37.420211    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:37.427042 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:37.427012    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:37.521578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:37.521541    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:37.528324 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:37.528288    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:37.611323 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:37.611289    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:37.611733 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:37.611705    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:37.612449 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:37.612429    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:37:37.646336 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:37.646132    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:37.646336 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:37.646188    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:37.653685 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:37.647169    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:37:37.653685 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:37.647313    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:37:40.351383 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.351317    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"]
Nov 04 07:37:40.356773 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.356748    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"
Nov 04 07:37:40.367120 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.367090    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-2a8c48-dockercfg-lvn6l\""
Nov 04 07:37:40.370278 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.370253    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"]
Nov 04 07:37:40.415257 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.415226    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-2a8c48\""
Nov 04 07:37:40.525382 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.522017    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvkml\" (UniqueName: \"kubernetes.io/projected/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-kube-api-access-xvkml\") pod \"payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6\" (UID: \"980bbac5-cf4e-4bbd-9fd6-60bf29548e37\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"
Nov 04 07:37:40.525382 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.522434    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6\" (UID: \"980bbac5-cf4e-4bbd-9fd6-60bf29548e37\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"
Nov 04 07:37:40.592872 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.592844    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"
Nov 04 07:37:40.623711 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.623621    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvkml\" (UniqueName: \"kubernetes.io/projected/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-kube-api-access-xvkml\") pod \"payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6\" (UID: \"980bbac5-cf4e-4bbd-9fd6-60bf29548e37\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"
Nov 04 07:37:40.623893 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.623712    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6\" (UID: \"980bbac5-cf4e-4bbd-9fd6-60bf29548e37\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"
Nov 04 07:37:40.624728 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.624703    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6\" (UID: \"980bbac5-cf4e-4bbd-9fd6-60bf29548e37\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"
Nov 04 07:37:40.661303 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.658435    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvkml\" (UniqueName: \"kubernetes.io/projected/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-kube-api-access-xvkml\") pod \"payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6\" (UID: \"980bbac5-cf4e-4bbd-9fd6-60bf29548e37\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"
Nov 04 07:37:40.673046 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.673011    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"
Nov 04 07:37:40.972269 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:40.972240    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"]
Nov 04 07:37:40.975970 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:37:40.975926    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod980bbac5_cf4e_4bbd_9fd6_60bf29548e37.slice/crio-05ca121a1309b465aa479c77fc184000bd39b70d0eb81def771aafd281afa4ac WatchSource:0}: Error finding container 05ca121a1309b465aa479c77fc184000bd39b70d0eb81def771aafd281afa4ac: Status 404 returned error can't find the container with id 05ca121a1309b465aa479c77fc184000bd39b70d0eb81def771aafd281afa4ac
Nov 04 07:37:41.716955 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:41.716912    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6" event={"ID":"980bbac5-cf4e-4bbd-9fd6-60bf29548e37","Type":"ContainerStarted","Data":"0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b"}
Nov 04 07:37:41.717350 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:41.717006    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6" event={"ID":"980bbac5-cf4e-4bbd-9fd6-60bf29548e37","Type":"ContainerStarted","Data":"05ca121a1309b465aa479c77fc184000bd39b70d0eb81def771aafd281afa4ac"}
Nov 04 07:37:41.717350 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:41.717143    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"
Nov 04 07:37:41.736916 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:41.736845    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6" podStartSLOduration=1.736826542 podStartE2EDuration="1.736826542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:41.735157673 +0000 UTC m=+1283.757372930" watchObservedRunningTime="2026-11-04 07:37:41.736826542 +0000 UTC m=+1283.759041803"
Nov 04 07:37:44.372873 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.372840    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"]
Nov 04 07:37:44.377935 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.377914    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.394420 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.394392    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"]
Nov 04 07:37:44.477654 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.477615    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.477846 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.477716    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.477846 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.477781    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2t8z\" (UniqueName: \"kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-kube-api-access-d2t8z\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.477846 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.477801    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.477846 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.477843    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/695aa56c-30fd-4351-abb6-b859413774e4-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.478048 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.477973    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.478048 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.477998    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.478048 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.478017    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.478184 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.478073    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/695aa56c-30fd-4351-abb6-b859413774e4-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.578995    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579233 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.579075    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579233 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.579104    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2t8z\" (UniqueName: \"kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-kube-api-access-d2t8z\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579233 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.579123    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579233 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.579171    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/695aa56c-30fd-4351-abb6-b859413774e4-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579487 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.579258    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579487 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.579284    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579487 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.579303    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579487 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.579331    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/695aa56c-30fd-4351-abb6-b859413774e4-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579683 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.579647    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579739 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.579679    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579739 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.579720    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.579893 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.579874    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.580181 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.580134    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/695aa56c-30fd-4351-abb6-b859413774e4-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.581649 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.581626    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.581883 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.581865    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/695aa56c-30fd-4351-abb6-b859413774e4-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.592786 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.592355    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.592786 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.592742    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2t8z\" (UniqueName: \"kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-kube-api-access-d2t8z\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.694128 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.694056    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:44.978615 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.978592    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"]
Nov 04 07:37:44.979495 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:37:44.979468    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod695aa56c_30fd_4351_abb6_b859413774e4.slice/crio-9525a327dbd9314ef0669c393f4e116eca32030fa66d1e89f15e06e7d5a0c5f0 WatchSource:0}: Error finding container 9525a327dbd9314ef0669c393f4e116eca32030fa66d1e89f15e06e7d5a0c5f0: Status 404 returned error can't find the container with id 9525a327dbd9314ef0669c393f4e116eca32030fa66d1e89f15e06e7d5a0c5f0
Nov 04 07:37:44.982588 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.982540    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:44.982684 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.982629    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:44.982684 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:44.982666    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:37:45.743934 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:45.743891    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl" event={"ID":"695aa56c-30fd-4351-abb6-b859413774e4","Type":"ContainerStarted","Data":"f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31"}
Nov 04 07:37:45.744381 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:45.743964    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl" event={"ID":"695aa56c-30fd-4351-abb6-b859413774e4","Type":"ContainerStarted","Data":"9525a327dbd9314ef0669c393f4e116eca32030fa66d1e89f15e06e7d5a0c5f0"}
Nov 04 07:37:45.771610 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:45.771562    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl" podStartSLOduration=1.771547601 podStartE2EDuration="1.771547601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:45.768800414 +0000 UTC m=+1287.791015685" watchObservedRunningTime="2026-11-04 07:37:45.771547601 +0000 UTC m=+1287.793762861"
Nov 04 07:37:46.694715 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:46.694680    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:46.696386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:46.696354    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 07:37:46.696478 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:46.696417    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl" podUID="695aa56c-30fd-4351-abb6-b859413774e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 07:37:47.695152 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:47.695113    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 07:37:47.695577 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:47.695192    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl" podUID="695aa56c-30fd-4351-abb6-b859413774e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 07:37:48.697389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:48.696732    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 07:37:48.697389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:48.696793    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl" podUID="695aa56c-30fd-4351-abb6-b859413774e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 07:37:49.694897 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:49.694859    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 07:37:49.695082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:49.694917    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl" podUID="695aa56c-30fd-4351-abb6-b859413774e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 07:37:50.695275 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:50.695232    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 07:37:50.695842 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:50.695302    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl" podUID="695aa56c-30fd-4351-abb6-b859413774e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 07:37:51.695387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:51.694748    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 07:37:51.695387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:51.694823    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl" podUID="695aa56c-30fd-4351-abb6-b859413774e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 07:37:52.726438 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:52.726408    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"
Nov 04 07:37:53.369519 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:53.368122    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:53.370313 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:53.370201    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:37:53.372063 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:37:53.372011    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:38:00.975233 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:00.975203    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"]
Nov 04 07:38:00.980462 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:00.980440    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:00.988519 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:00.988446    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-812032\"/\"agc-cred-aa18fa2b2c2f\""
Nov 04 07:38:00.989937 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:00.989913    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-812032\"/\"payload-processing-external-model-plugins-e2e-extmod-b-812032\""
Nov 04 07:38:00.990471 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:00.990244    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-812032\"/\"openshift-service-ca.crt\""
Nov 04 07:38:00.990471 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:00.990305    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-812032\"/\"routing-overlay\""
Nov 04 07:38:00.990471 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:00.990398    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-812032\"/\"kube-root-ca.crt\""
Nov 04 07:38:00.993426 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:00.993406    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"]
Nov 04 07:38:01.033818 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.033781    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-812032-8489sj855\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") " pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.034008 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.033916    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-812032-8489sj855\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") " pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.034151 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.034122    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-812032-8489sj855\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") " pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.034266 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.034247    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dmzv\" (UniqueName: \"kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-kube-api-access-9dmzv\") pod \"payload-processing-external-model-e2e-extmod-b-812032-8489sj855\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") " pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.041491 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.041466    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"]
Nov 04 07:38:01.045509 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.045491    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.061144 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.061115    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"]
Nov 04 07:38:01.098533 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.098501    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"]
Nov 04 07:38:01.103535 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.103511    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.110154 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.110089    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"]
Nov 04 07:38:01.136091 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136054    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.136264 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136113    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dmzv\" (UniqueName: \"kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-kube-api-access-9dmzv\") pod \"payload-processing-external-model-e2e-extmod-b-812032-8489sj855\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") " pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.136264 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136182    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgk2c\" (UniqueName: \"kubernetes.io/projected/614988be-64cf-4a39-b201-ff69a7f7fb19-kube-api-access-xgk2c\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.136264 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136219    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-812032-8489sj855\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") " pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.136264 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136250    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.136520 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136278    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/614988be-64cf-4a39-b201-ff69a7f7fb19-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.136520 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136354    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-812032-8489sj855\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") " pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.136520 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136422    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/614988be-64cf-4a39-b201-ff69a7f7fb19-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.136679 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136542    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.136679 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136571    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.136679 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136607    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/614988be-64cf-4a39-b201-ff69a7f7fb19-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.136832 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136685    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-812032-8489sj855\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") " pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.136832 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.136728    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.139296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.137821    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-812032-8489sj855\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") " pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.139296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.138604    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-812032-8489sj855\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") " pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.180427 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.174322    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dmzv\" (UniqueName: \"kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-kube-api-access-9dmzv\") pod \"payload-processing-external-model-e2e-extmod-b-812032-8489sj855\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") " pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.203494 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.203322    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-812032-8489sj855\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") " pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.237533 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.237441    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.237533 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.237481    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.237533 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.237516    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgk2c\" (UniqueName: \"kubernetes.io/projected/614988be-64cf-4a39-b201-ff69a7f7fb19-kube-api-access-xgk2c\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.237819 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.237712    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.237819 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.237762    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.237819 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.237787    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/614988be-64cf-4a39-b201-ff69a7f7fb19-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.237962 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.237868    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.237962 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.237920    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.238063 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238048    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/614988be-64cf-4a39-b201-ff69a7f7fb19-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.238122 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238076    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.238122 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238114    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lffgz\" (UniqueName: \"kubernetes.io/projected/5be66e14-e024-4afe-8e07-9cd3cc138ea5-kube-api-access-lffgz\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.238221 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238198    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.238274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238223    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.238274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238247    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/614988be-64cf-4a39-b201-ff69a7f7fb19-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.238410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238289    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.238410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238311    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.238410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238344    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.238410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238405    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.238631 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238199    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.238631 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238599    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.238734 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238711    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/614988be-64cf-4a39-b201-ff69a7f7fb19-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.238802 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238770    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.238884 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.238863    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.240721 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.240692    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/614988be-64cf-4a39-b201-ff69a7f7fb19-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.241289 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.241268    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/614988be-64cf-4a39-b201-ff69a7f7fb19-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.249902 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.249881    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgk2c\" (UniqueName: \"kubernetes.io/projected/614988be-64cf-4a39-b201-ff69a7f7fb19-kube-api-access-xgk2c\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.250003 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.249982    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/614988be-64cf-4a39-b201-ff69a7f7fb19-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-g5rgr\" (UID: \"614988be-64cf-4a39-b201-ff69a7f7fb19\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.296072 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.296034    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.339771 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.339738    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.339933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.339786    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.339933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.339840    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.339933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.339859    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.339933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.339892    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.339933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.339915    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.340099 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.339945    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.340099 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.339990    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.340099 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.340020    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lffgz\" (UniqueName: \"kubernetes.io/projected/5be66e14-e024-4afe-8e07-9cd3cc138ea5-kube-api-access-lffgz\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.340552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.340525    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.340685 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.340546    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.341157 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.341133    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.341253 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.341134    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.341253 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.341137    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.342933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.342914    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.343308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.343277    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.353231 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.353195    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lffgz\" (UniqueName: \"kubernetes.io/projected/5be66e14-e024-4afe-8e07-9cd3cc138ea5-kube-api-access-lffgz\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.353355 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.353316    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5be66e14-e024-4afe-8e07-9cd3cc138ea5-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-788ls\" (UID: \"5be66e14-e024-4afe-8e07-9cd3cc138ea5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.369742 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.369502    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:01.434661 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.434302    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:01.594017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.593974    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"]
Nov 04 07:38:01.750280 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:38:01.750231    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614988be_64cf_4a39_b201_ff69a7f7fb19.slice/crio-9a0eecc60dc3d962ed3668e41cb66f4006134d98fe8b94eb627e69f185d2eafe WatchSource:0}: Error finding container 9a0eecc60dc3d962ed3668e41cb66f4006134d98fe8b94eb627e69f185d2eafe: Status 404 returned error can't find the container with id 9a0eecc60dc3d962ed3668e41cb66f4006134d98fe8b94eb627e69f185d2eafe
Nov 04 07:38:01.754510 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.753611    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:01.754510 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.753699    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:01.754510 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.753747    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:01.755438 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.755263    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"]
Nov 04 07:38:01.801930 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.801667    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"]
Nov 04 07:38:01.804173 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:38:01.804136    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be66e14_e024_4afe_8e07_9cd3cc138ea5.slice/crio-3a95fe66e582e7c907f772ecb512f5a4cf5c3c66bec1910b5b915ca89ae82fad WatchSource:0}: Error finding container 3a95fe66e582e7c907f772ecb512f5a4cf5c3c66bec1910b5b915ca89ae82fad: Status 404 returned error can't find the container with id 3a95fe66e582e7c907f772ecb512f5a4cf5c3c66bec1910b5b915ca89ae82fad
Nov 04 07:38:01.807881 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.807844    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:01.807981 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.807931    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:01.808024 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.808003    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:01.919628 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.919553    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" event={"ID":"5be66e14-e024-4afe-8e07-9cd3cc138ea5","Type":"ContainerStarted","Data":"efb14af60a889f818b41b7e72af0841943744fda8347116a5b62b390f2df77a3"}
Nov 04 07:38:01.919934 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.919908    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" event={"ID":"5be66e14-e024-4afe-8e07-9cd3cc138ea5","Type":"ContainerStarted","Data":"3a95fe66e582e7c907f772ecb512f5a4cf5c3c66bec1910b5b915ca89ae82fad"}
Nov 04 07:38:01.924730 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.923960    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855" event={"ID":"9cc5b6e5-a888-4e86-8f9d-18fc9499167c","Type":"ContainerStarted","Data":"10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774"}
Nov 04 07:38:01.924730 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.924016    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855" event={"ID":"9cc5b6e5-a888-4e86-8f9d-18fc9499167c","Type":"ContainerStarted","Data":"6af7d1d90305cd614eca38590c044c1df322b00ebc9c9d81310d5e5b13a54208"}
Nov 04 07:38:01.924730 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.924682    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:01.927667 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.927032    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" event={"ID":"614988be-64cf-4a39-b201-ff69a7f7fb19","Type":"ContainerStarted","Data":"324ffc7a2b8697bb039a804686554cd6133b0beb24949902cba4a5d96c0e8628"}
Nov 04 07:38:01.927667 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.927071    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" event={"ID":"614988be-64cf-4a39-b201-ff69a7f7fb19","Type":"ContainerStarted","Data":"9a0eecc60dc3d962ed3668e41cb66f4006134d98fe8b94eb627e69f185d2eafe"}
Nov 04 07:38:01.958170 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.958099    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podStartSLOduration=0.958082113 podStartE2EDuration="958.082113ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:01.953972198 +0000 UTC m=+1303.976187462" watchObservedRunningTime="2026-11-04 07:38:01.958082113 +0000 UTC m=+1303.980297374"
Nov 04 07:38:01.980661 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:01.980607    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podStartSLOduration=0.98058699 podStartE2EDuration="980.58699ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:01.976907975 +0000 UTC m=+1303.999123272" watchObservedRunningTime="2026-11-04 07:38:01.98058699 +0000 UTC m=+1304.002802251"
Nov 04 07:38:02.003456 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:02.003213    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855" podStartSLOduration=2.003192889 podStartE2EDuration="2.003192889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:02.000618457 +0000 UTC m=+1304.022833743" watchObservedRunningTime="2026-11-04 07:38:02.003192889 +0000 UTC m=+1304.025408150"
Nov 04 07:38:02.370989 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:02.370869    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:02.372250 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:02.372225    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 07:38:02.372402 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:02.372278    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 07:38:02.465385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:02.452396    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:02.491741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:02.491704    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 07:38:02.492399 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:02.492012    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 07:38:03.381092 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:03.381046    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 07:38:03.381780 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:03.381726    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 07:38:03.492390 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:03.491760    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 07:38:03.492390 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:03.491824    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 07:38:03.792387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:03.787480    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:03.840397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:03.835962    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:03.866397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:03.866107    5626 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:04.190200 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.189518    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" podUID="573aaebc-ca66-4bf2-84a5-2057a5e77b87" containerName="main" probeResult="failure" output="Get \"https://10.134.0.38:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:04.376734 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.376696    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 07:38:04.377021 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.376991    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 07:38:04.446448 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.443473    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 07:38:04.446448 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.443544    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 07:38:04.489389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.477450    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:04.575387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.565463    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:04.696597 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.696487    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:04.755389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.753465    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:04.790520 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.790467    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:04.791896 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.791860    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:04.792473 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.792455    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:38:04.839509 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.839460    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:04.840508 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:04.840486    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:38:05.200345 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.199061    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" podUID="573aaebc-ca66-4bf2-84a5-2057a5e77b87" containerName="main" probeResult="failure" output="Get \"https://10.134.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:05.222508 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.205760    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:38:05.493442 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.489267    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 07:38:05.493442 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.491333    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:05.493442 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.492549    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:38:05.499880 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.489345    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 07:38:05.545075 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.545026    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 07:38:05.545320 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.545294    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 07:38:05.599409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.583492    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:05.599409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.594524    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:38:05.727389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.725572    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:05.751442 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.749388    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:38:05.761490 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.761446    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 04 07:38:05.762942 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.762920    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:38:05.797397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.780877    5626 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:05.797694 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.791606    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:05.798405 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.798379    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:38:05.819927 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.799388    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 04 07:38:05.843905 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:05.843858    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 07:38:06.213389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.209336    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" podUID="573aaebc-ca66-4bf2-84a5-2057a5e77b87" containerName="main" probeResult="failure" output="Get \"https://10.134.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:06.398388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.397385    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 07:38:06.398388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.397446    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 07:38:06.451015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.450966    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 07:38:06.451323 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.451291    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 07:38:06.504441 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.493140    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:06.529400 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.508525    5626 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": context deadline exceeded"
Nov 04 07:38:06.586682 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.586643    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 04 07:38:06.606387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.604912    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"]
Nov 04 07:38:06.647389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.631202    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:06.683392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.661312    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-023307-openshift-default-dockercfg-jvkfh\""
Nov 04 07:38:06.747810 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.730445    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:06.766918 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.766785    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:06.925653 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:38:06.925596    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:38:06.929698 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.929660    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"]
Nov 04 07:38:06.937388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.933642    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 04 07:38:06.951650 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:06.951591    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:07.060436 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.051562    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 07:38:07.246483 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.235921    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" podUID="573aaebc-ca66-4bf2-84a5-2057a5e77b87" containerName="main" probeResult="failure" output="Get \"https://10.134.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:07.470946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.398952    5626 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:07.470946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.409890    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 07:38:07.470946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.445915    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 07:38:07.470946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.420048    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-data\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.470946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.446064    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-socket\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.470946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.446102    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-envoy\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.470946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.446171    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-token\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.470946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.446224    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istiod-ca-cert\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.470946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.446259    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vt8h\" (UniqueName: \"kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-kube-api-access-6vt8h\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.470946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.446498    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-podinfo\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.470946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.446621    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-credential-socket\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.470946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.446669    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-certs\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.580557 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.580505    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:07.730451 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.612270    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-credential-socket\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.730451 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.703124    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-certs\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.730451 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.703237    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-data\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.730451 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.703297    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-socket\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.730451 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.703323    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-envoy\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.730451 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.703409    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-token\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.730451 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.703456    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istiod-ca-cert\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.730451 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.703490    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vt8h\" (UniqueName: \"kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-kube-api-access-6vt8h\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.730451 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.703717    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-podinfo\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.848387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.792118    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-data\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.848387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.612735    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-credential-socket\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.848387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.792483    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-certs\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:07.848387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.613094    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 07:38:07.848387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.792556    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 07:38:07.848387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.794659    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:07.848387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.639634    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 04 07:38:07.970387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.966659    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:07.994509 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.974784    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 04 07:38:08.161548 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.967613    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istiod-ca-cert\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:08.161810 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.968284    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:08.162098 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:07.966859    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-socket\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:08.162237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.094816    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-envoy\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:08.162338 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.095380    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 07:38:08.162501 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.129438    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-token\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:08.162574 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.137241    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-podinfo\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:08.417387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.388218    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 07:38:08.417387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.388284    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 07:38:08.459857 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.459809    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" podUID="573aaebc-ca66-4bf2-84a5-2057a5e77b87" containerName="main" probeResult="failure" output="Get \"https://10.134.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:08.473036 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.472993    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 07:38:08.477385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.473266    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 07:38:08.486395 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.479136    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vt8h\" (UniqueName: \"kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-kube-api-access-6vt8h\") pod \"e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") " pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:08.552415 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.552353    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:08.725449 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.724893    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:08.799387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.795805    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 04 07:38:08.799387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.796235    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Nov 04 07:38:08.984437 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:08.983507    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:09.165508 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.163448    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:09.165508 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.163852    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 04 07:38:09.165508 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.165010    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:09.413388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.402688    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 07:38:09.413388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.402755    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 07:38:09.413388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.403347    5626 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:09.520388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.476430    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" podUID="573aaebc-ca66-4bf2-84a5-2057a5e77b87" containerName="main" probeResult="failure" output="Get \"https://10.134.0.38:8000/ready\": context deadline exceeded"
Nov 04 07:38:09.791078 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.791037    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 07:38:09.793154 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.793120    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 07:38:09.854399 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.853925    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:09.951389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.903432    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Nov 04 07:38:09.951389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.904543    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:09.993394 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:09.992835    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 07:38:10.112658 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:38:10.098048    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc915aa8b_a06f_4446_a012_dd9c0f383b7f.slice/crio-a026b3110aea4d2ab8416acecd2082478f7115016e64be4a90f0cb61ab65a609 WatchSource:0}: Error finding container a026b3110aea4d2ab8416acecd2082478f7115016e64be4a90f0cb61ab65a609: Status 404 returned error can't find the container with id a026b3110aea4d2ab8416acecd2082478f7115016e64be4a90f0cb61ab65a609
Nov 04 07:38:10.112658 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.108379    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:10.112658 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.108468    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:10.112658 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.108508    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:10.167213 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.165596    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:10.167213 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.166005    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:10.167591 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.167542    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:10.192860 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.192830    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"]
Nov 04 07:38:10.332544 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.331433    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:38:10.374455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.373722    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 07:38:10.374455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.373781    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 07:38:10.456538 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.456497    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 07:38:10.456700 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.456560    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 07:38:10.664998 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.664965    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:38:10.863512 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.862470    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:10.910435 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:10.908794    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:11.013498 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.013449    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 07:38:11.171280 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.168494    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:11.171280 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.168916    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 04 07:38:11.171280 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.169211    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 04 07:38:11.197242 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.197194    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" event={"ID":"c915aa8b-a06f-4446-a012-dd9c0f383b7f","Type":"ContainerStarted","Data":"1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8"}
Nov 04 07:38:11.197571 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.197531    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" event={"ID":"c915aa8b-a06f-4446-a012-dd9c0f383b7f","Type":"ContainerStarted","Data":"a026b3110aea4d2ab8416acecd2082478f7115016e64be4a90f0cb61ab65a609"}
Nov 04 07:38:11.249508 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.249352    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" podStartSLOduration=5.249330111 podStartE2EDuration="5.249330111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:11.243869393 +0000 UTC m=+1313.266084657" watchObservedRunningTime="2026-11-04 07:38:11.249330111 +0000 UTC m=+1313.271545374"
Nov 04 07:38:11.333339 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.333304    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:38:11.396387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.378630    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 07:38:11.406387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.403996    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 07:38:11.439658 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.439558    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 07:38:11.439953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.439903    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 07:38:11.518845 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.518814    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:38:11.557459 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.557426    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:11.562798 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.562765    5626 patch_prober.go:29] interesting pod/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 07:38:11.571149 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.571111    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 07:38:11.667324 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.666496    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" podUID="573aaebc-ca66-4bf2-84a5-2057a5e77b87" containerName="main" probeResult="failure" output="Get \"https://10.134.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:11.802865 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.713913    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:38:11.944427 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.944392    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:38:12.004387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:11.997870    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:38:12.017895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:12.017850    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:38:12.385440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:12.376182    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 07:38:12.385440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:12.376247    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 07:38:12.435799 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:12.435760    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 07:38:12.436084 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:12.436052    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 07:38:12.563729 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:12.563686    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:13.253639 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.253602    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"]
Nov 04 07:38:13.260139 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.260112    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:13.260560 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.260433    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"
Nov 04 07:38:13.260864 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.260545    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:13.276648 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.270793    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-be2a74-dockercfg-fn44v\""
Nov 04 07:38:13.276648 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.270895    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-be2a74\""
Nov 04 07:38:13.276648 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.276190    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"]
Nov 04 07:38:13.296617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.291485    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"]
Nov 04 07:38:13.324130 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.323623    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"
Nov 04 07:38:13.349585 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.349542    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"]
Nov 04 07:38:13.368132 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.366884    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcf5g\" (UniqueName: \"kubernetes.io/projected/2702960b-b62a-4bfd-8457-13eb82b88043-kube-api-access-kcf5g\") pod \"payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t\" (UID: \"2702960b-b62a-4bfd-8457-13eb82b88043\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"
Nov 04 07:38:13.368132 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.367083    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2702960b-b62a-4bfd-8457-13eb82b88043-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t\" (UID: \"2702960b-b62a-4bfd-8457-13eb82b88043\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"
Nov 04 07:38:13.371292 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.370862    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 07:38:13.371292 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.370914    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 07:38:13.435558 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.435431    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 07:38:13.435558 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.435491    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 07:38:13.469442 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.468304    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f351198-a3ef-4598-a748-26b51b757a0a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg\" (UID: \"0f351198-a3ef-4598-a748-26b51b757a0a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"
Nov 04 07:38:13.469442 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.468471    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2702960b-b62a-4bfd-8457-13eb82b88043-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t\" (UID: \"2702960b-b62a-4bfd-8457-13eb82b88043\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"
Nov 04 07:38:13.469442 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.468596    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs4lh\" (UniqueName: \"kubernetes.io/projected/0f351198-a3ef-4598-a748-26b51b757a0a-kube-api-access-bs4lh\") pod \"payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg\" (UID: \"0f351198-a3ef-4598-a748-26b51b757a0a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"
Nov 04 07:38:13.469442 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.468716    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcf5g\" (UniqueName: \"kubernetes.io/projected/2702960b-b62a-4bfd-8457-13eb82b88043-kube-api-access-kcf5g\") pod \"payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t\" (UID: \"2702960b-b62a-4bfd-8457-13eb82b88043\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"
Nov 04 07:38:13.470550 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.470494    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2702960b-b62a-4bfd-8457-13eb82b88043-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t\" (UID: \"2702960b-b62a-4bfd-8457-13eb82b88043\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"
Nov 04 07:38:13.537317 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.537247    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcf5g\" (UniqueName: \"kubernetes.io/projected/2702960b-b62a-4bfd-8457-13eb82b88043-kube-api-access-kcf5g\") pod \"payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t\" (UID: \"2702960b-b62a-4bfd-8457-13eb82b88043\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"
Nov 04 07:38:13.581562 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.570301    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f351198-a3ef-4598-a748-26b51b757a0a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg\" (UID: \"0f351198-a3ef-4598-a748-26b51b757a0a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"
Nov 04 07:38:13.581562 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.570563    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs4lh\" (UniqueName: \"kubernetes.io/projected/0f351198-a3ef-4598-a748-26b51b757a0a-kube-api-access-bs4lh\") pod \"payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg\" (UID: \"0f351198-a3ef-4598-a748-26b51b757a0a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"
Nov 04 07:38:13.581562 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.571872    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f351198-a3ef-4598-a748-26b51b757a0a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg\" (UID: \"0f351198-a3ef-4598-a748-26b51b757a0a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"
Nov 04 07:38:13.593989 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.593656    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"
Nov 04 07:38:13.644318 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.644241    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs4lh\" (UniqueName: \"kubernetes.io/projected/0f351198-a3ef-4598-a748-26b51b757a0a-kube-api-access-bs4lh\") pod \"payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg\" (UID: \"0f351198-a3ef-4598-a748-26b51b757a0a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"
Nov 04 07:38:13.657831 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:13.657795    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"
Nov 04 07:38:14.071236 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:14.058147    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:38:14.438527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:14.438413    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"]
Nov 04 07:38:14.445081 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:14.445045    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"]
Nov 04 07:38:15.436385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.435638    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:38:15.436385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.435705    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:15.449352 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.448954    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:38:15.449352 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.449016    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:15.494088 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.494043    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t" event={"ID":"2702960b-b62a-4bfd-8457-13eb82b88043","Type":"ContainerStarted","Data":"aabdfcd4c12ce7ce6d3343867c9583795c4f69d1ea652a5cbd376bc9fcf4f84b"}
Nov 04 07:38:15.494251 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.494114    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t" event={"ID":"2702960b-b62a-4bfd-8457-13eb82b88043","Type":"ContainerStarted","Data":"d542cbb4a7f5252ccfc9ad40f3f688cdef2578fe858abd1180233d0c4b521ebf"}
Nov 04 07:38:15.495199 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.495175    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"
Nov 04 07:38:15.499450 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.498609    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg" event={"ID":"0f351198-a3ef-4598-a748-26b51b757a0a","Type":"ContainerStarted","Data":"a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1"}
Nov 04 07:38:15.499450 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.498656    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"
Nov 04 07:38:15.499450 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.498681    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg" event={"ID":"0f351198-a3ef-4598-a748-26b51b757a0a","Type":"ContainerStarted","Data":"e07e81032e519fe1ed5d5ea73d370b54cacce22d212a594e5961e33f42bd5e55"}
Nov 04 07:38:15.582795 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.561043    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t" podStartSLOduration=2.561019987 podStartE2EDuration="2.561019987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:15.531788256 +0000 UTC m=+1317.554003614" watchObservedRunningTime="2026-11-04 07:38:15.561019987 +0000 UTC m=+1317.583235248"
Nov 04 07:38:15.582795 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.567709    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"]
Nov 04 07:38:15.582795 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.575476    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"
Nov 04 07:38:15.582795 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.576814    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"]
Nov 04 07:38:15.584988 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.584898    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg" podStartSLOduration=2.5848760950000003 podStartE2EDuration="2.584876095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:15.566201621 +0000 UTC m=+1317.588416863" watchObservedRunningTime="2026-11-04 07:38:15.584876095 +0000 UTC m=+1317.607091360"
Nov 04 07:38:15.587237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.587193    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-023307-dockercfg-jsbwr\""
Nov 04 07:38:15.591559 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.591529    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-023307\""
Nov 04 07:38:15.592221 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.592186    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aceda029-76df-4e8a-ba59-115b11c12b7f-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb\" (UID: \"aceda029-76df-4e8a-ba59-115b11c12b7f\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"
Nov 04 07:38:15.593268 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.592756    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2bxr\" (UniqueName: \"kubernetes.io/projected/aceda029-76df-4e8a-ba59-115b11c12b7f-kube-api-access-k2bxr\") pod \"payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb\" (UID: \"aceda029-76df-4e8a-ba59-115b11c12b7f\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"
Nov 04 07:38:15.711431 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.711176    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2bxr\" (UniqueName: \"kubernetes.io/projected/aceda029-76df-4e8a-ba59-115b11c12b7f-kube-api-access-k2bxr\") pod \"payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb\" (UID: \"aceda029-76df-4e8a-ba59-115b11c12b7f\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"
Nov 04 07:38:15.711431 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.711266    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aceda029-76df-4e8a-ba59-115b11c12b7f-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb\" (UID: \"aceda029-76df-4e8a-ba59-115b11c12b7f\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"
Nov 04 07:38:15.769028 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.768989    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aceda029-76df-4e8a-ba59-115b11c12b7f-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb\" (UID: \"aceda029-76df-4e8a-ba59-115b11c12b7f\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"
Nov 04 07:38:15.816411 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.812332    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2bxr\" (UniqueName: \"kubernetes.io/projected/aceda029-76df-4e8a-ba59-115b11c12b7f-kube-api-access-k2bxr\") pod \"payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb\" (UID: \"aceda029-76df-4e8a-ba59-115b11c12b7f\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"
Nov 04 07:38:15.935073 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:15.934971    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"
Nov 04 07:38:16.309132 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:16.309093    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"]
Nov 04 07:38:16.310585 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:38:16.310558    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaceda029_76df_4e8a_ba59_115b11c12b7f.slice/crio-a44aff20b37182e095866d9874c33343cfbbce74bf5c167e0354f534529f4068 WatchSource:0}: Error finding container a44aff20b37182e095866d9874c33343cfbbce74bf5c167e0354f534529f4068: Status 404 returned error can't find the container with id a44aff20b37182e095866d9874c33343cfbbce74bf5c167e0354f534529f4068
Nov 04 07:38:16.371378 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:16.370310    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:16.436232 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:16.435758    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:16.438510 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:16.437569    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:38:16.438510 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:16.437627    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:16.452406 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:16.452111    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:38:16.452406 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:16.452177    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:16.512710 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:16.511998    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb" event={"ID":"aceda029-76df-4e8a-ba59-115b11c12b7f","Type":"ContainerStarted","Data":"11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160"}
Nov 04 07:38:16.512710 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:16.512056    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb" event={"ID":"aceda029-76df-4e8a-ba59-115b11c12b7f","Type":"ContainerStarted","Data":"a44aff20b37182e095866d9874c33343cfbbce74bf5c167e0354f534529f4068"}
Nov 04 07:38:16.512710 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:16.512648    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"
Nov 04 07:38:16.544385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:16.542540    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb" podStartSLOduration=1.542521594 podStartE2EDuration="1.542521594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:16.538646521 +0000 UTC m=+1318.560861779" watchObservedRunningTime="2026-11-04 07:38:16.542521594 +0000 UTC m=+1318.564736856"
Nov 04 07:38:17.491520 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:17.491476    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:38:17.492302 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:17.492266    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:17.526910 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:17.524540    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:38:17.526910 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:17.524606    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:18.537964 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:18.537495    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:38:18.537964 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:18.537561    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:18.614539 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:18.578455    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:38:18.614539 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:18.578524    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:38:18.921120 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:18.920716    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:18.923437 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:18.923333    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls"
Nov 04 07:38:19.020023 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.016827    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:19.098181 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.025589    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr"
Nov 04 07:38:19.580882 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.580845    5626 generic.go:356] "Generic (PLEG): container finished" podID="fea08504-e401-4132-8b45-1f9b36646205" containerID="1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea" exitCode=0
Nov 04 07:38:19.581296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.580919    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" event={"ID":"fea08504-e401-4132-8b45-1f9b36646205","Type":"ContainerDied","Data":"1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea"}
Nov 04 07:38:19.581396 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.581381    5626 scope.go:122] "RemoveContainer" containerID="1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea"
Nov 04 07:38:19.582797 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.582763    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:19.582911 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.582821    5626 generic.go:356] "Generic (PLEG): container finished" podID="db38048a-323a-4ab4-a60a-a394546f78f6" containerID="bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b" exitCode=0
Nov 04 07:38:19.582911 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.582836    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:19.582911 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.582870    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:19.582911 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.582891    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" event={"ID":"db38048a-323a-4ab4-a60a-a394546f78f6","Type":"ContainerDied","Data":"bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b"}
Nov 04 07:38:19.583623 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.583608    5626 scope.go:122] "RemoveContainer" containerID="bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b"
Nov 04 07:38:19.584844 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.584781    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:19.584946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.584887    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:19.584946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:19.584933    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:20.600285 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:20.600249    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" event={"ID":"db38048a-323a-4ab4-a60a-a394546f78f6","Type":"ContainerStarted","Data":"16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886"}
Nov 04 07:38:20.602148 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:20.602125    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" event={"ID":"fea08504-e401-4132-8b45-1f9b36646205","Type":"ContainerStarted","Data":"372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6"}
Nov 04 07:38:21.422388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:21.419716    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:38:21.432547 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:21.430486    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 07:38:21.432547 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:21.430547    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" podUID="fea08504-e401-4132-8b45-1f9b36646205" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 07:38:21.535387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:21.523692    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:38:21.535387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:21.523871    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 07:38:21.535387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:21.523914    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 07:38:22.419655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:22.419611    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 07:38:22.420177 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:22.419673    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" podUID="fea08504-e401-4132-8b45-1f9b36646205" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 07:38:22.521545 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:22.521509    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 07:38:22.521719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:22.521563    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 07:38:22.654415 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:22.648875    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:38:22.654415 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:22.649007    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:38:23.420290 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:23.420253    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 07:38:23.420811 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:23.420455    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" podUID="fea08504-e401-4132-8b45-1f9b36646205" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 07:38:23.522204 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:23.521705    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 07:38:23.522204 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:23.521762    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 07:38:24.419590 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:24.419545    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 07:38:24.419788 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:24.419611    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" podUID="fea08504-e401-4132-8b45-1f9b36646205" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 07:38:24.522686 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:24.522595    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 07:38:24.522686 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:24.522656    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 07:38:25.420489 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:25.420279    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 07:38:25.420489 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:25.420353    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" podUID="fea08504-e401-4132-8b45-1f9b36646205" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 07:38:25.522311 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:25.522274    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 07:38:25.522508 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:25.522352    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 07:38:26.424463 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:26.424434    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:38:26.425572 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:26.425550    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:38:26.515519 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:26.515493    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"
Nov 04 07:38:26.528142 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:26.528103    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:38:26.529456 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:26.529428    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:38:27.540707 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:27.540681    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"
Nov 04 07:38:28.576199 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:28.576173    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"
Nov 04 07:38:31.440882 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.440845    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"]
Nov 04 07:38:31.444273 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.444254    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.448060 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:38:31.448031    5626 status_manager.go:1045] "Failed to get status for pod" err="pods \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" is forbidden: User \"system:node:ip-10-0-142-197.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-197.us-east-2.compute.internal' and this object" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.450658 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:38:31.450634    5626 reflector.go:204] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-extmod-a-5c1e58-openshift-default-dockercfg-z7kk4\" is forbidden: User \"system:node:ip-10-0-142-197.us-east-2.compute.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-197.us-east-2.compute.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-5c1e58-openshift-default-dockercfg-z7kk4\"" type="*v1.Secret"
Nov 04 07:38:31.578008 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.555120    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-podinfo\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.578499 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.578463    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-socket\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.578765 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.578745    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/222aacaf-0c37-4aad-91ad-5607a50820ce-istiod-ca-cert\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.578936 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.578909    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-token\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.579093 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.579069    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-envoy\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.579320 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.579305    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-data\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.579594 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.579577    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmf4h\" (UniqueName: \"kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-kube-api-access-nmf4h\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.579748 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.579733    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-credential-socket\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.579896 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.579879    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-certs\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.616058 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.616013    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"]
Nov 04 07:38:31.681603 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.681572    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/222aacaf-0c37-4aad-91ad-5607a50820ce-istiod-ca-cert\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.681815 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.681616    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-token\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.681815 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.681637    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-envoy\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.681815 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.681689    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-data\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.681815 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.681749    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmf4h\" (UniqueName: \"kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-kube-api-access-nmf4h\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.681815 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.681787    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-credential-socket\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.682093 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.681824    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-certs\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.682093 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.681892    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-podinfo\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.682093 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.681969    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-socket\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.682256 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.682170    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-data\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.682304 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.682250    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-certs\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.682304 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.682286    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-credential-socket\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.682441 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.682316    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-socket\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.682441 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.682348    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/222aacaf-0c37-4aad-91ad-5607a50820ce-istiod-ca-cert\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.683984 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.683960    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-envoy\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.684279 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.684261    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-podinfo\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.696942 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.696887    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-token\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:31.696942 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:31.696902    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmf4h\" (UniqueName: \"kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-kube-api-access-nmf4h\") pod \"e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") " pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:32.744110 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:32.744078    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-5c1e58-openshift-default-dockercfg-z7kk4\""
Nov 04 07:38:32.749044 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:32.749018    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:32.972732 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:32.972703    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"]
Nov 04 07:38:32.976349 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:32.976326    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:32.989658 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:32.989635    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"]
Nov 04 07:38:32.990318 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:32.990297    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:38:32.991251 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:32.991232    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:38:32.991424 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:32.991261    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:38:32.991424 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:32.991289    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:38:33.000477 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.000414    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4hlg\" (UniqueName: \"kubernetes.io/projected/8c968ff1-4226-4571-8726-0c7f2f463e11-kube-api-access-h4hlg\") pod \"maas-api-revoke-sub-c3eb9c5e-zwmv8\" (UID: \"8c968ff1-4226-4571-8726-0c7f2f463e11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:33.000583 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.000521    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c3eb9c5e-zwmv8\" (UID: \"8c968ff1-4226-4571-8726-0c7f2f463e11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:33.000649 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.000599    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c3eb9c5e-zwmv8\" (UID: \"8c968ff1-4226-4571-8726-0c7f2f463e11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:33.076334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.076305    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"]
Nov 04 07:38:33.076545 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:38:33.076525    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222aacaf_0c37_4aad_91ad_5607a50820ce.slice/crio-84321167d6f661a25828aa83b1bacbaea0a8fd2ae53c8e93c1ba2c6e4865ab27 WatchSource:0}: Error finding container 84321167d6f661a25828aa83b1bacbaea0a8fd2ae53c8e93c1ba2c6e4865ab27: Status 404 returned error can't find the container with id 84321167d6f661a25828aa83b1bacbaea0a8fd2ae53c8e93c1ba2c6e4865ab27
Nov 04 07:38:33.079514 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.079481    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:33.079626 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.079543    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:33.079626 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.079574    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:33.101550 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.101519    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c3eb9c5e-zwmv8\" (UID: \"8c968ff1-4226-4571-8726-0c7f2f463e11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:33.101676 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.101619    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c3eb9c5e-zwmv8\" (UID: \"8c968ff1-4226-4571-8726-0c7f2f463e11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:33.101835 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.101820    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4hlg\" (UniqueName: \"kubernetes.io/projected/8c968ff1-4226-4571-8726-0c7f2f463e11-kube-api-access-h4hlg\") pod \"maas-api-revoke-sub-c3eb9c5e-zwmv8\" (UID: \"8c968ff1-4226-4571-8726-0c7f2f463e11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:33.102706 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.102683    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c3eb9c5e-zwmv8\" (UID: \"8c968ff1-4226-4571-8726-0c7f2f463e11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:33.104429 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.104409    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c3eb9c5e-zwmv8\" (UID: \"8c968ff1-4226-4571-8726-0c7f2f463e11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:33.120232 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.120204    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4hlg\" (UniqueName: \"kubernetes.io/projected/8c968ff1-4226-4571-8726-0c7f2f463e11-kube-api-access-h4hlg\") pod \"maas-api-revoke-sub-c3eb9c5e-zwmv8\" (UID: \"8c968ff1-4226-4571-8726-0c7f2f463e11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:33.292108 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.292080    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:33.577784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.577760    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"]
Nov 04 07:38:33.578441 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:38:33.578416    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c968ff1_4226_4571_8726_0c7f2f463e11.slice/crio-b545bff38e516807535712f60803da224bf51f31ca29d3ab5c2e935c72a802fd WatchSource:0}: Error finding container b545bff38e516807535712f60803da224bf51f31ca29d3ab5c2e935c72a802fd: Status 404 returned error can't find the container with id b545bff38e516807535712f60803da224bf51f31ca29d3ab5c2e935c72a802fd
Nov 04 07:38:33.580437 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.580417    5626 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 07:38:33.774507 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.774470    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8" event={"ID":"8c968ff1-4226-4571-8726-0c7f2f463e11","Type":"ContainerStarted","Data":"b545bff38e516807535712f60803da224bf51f31ca29d3ab5c2e935c72a802fd"}
Nov 04 07:38:33.776556 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.776525    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" event={"ID":"222aacaf-0c37-4aad-91ad-5607a50820ce","Type":"ContainerStarted","Data":"2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087"}
Nov 04 07:38:33.776707 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.776577    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" event={"ID":"222aacaf-0c37-4aad-91ad-5607a50820ce","Type":"ContainerStarted","Data":"84321167d6f661a25828aa83b1bacbaea0a8fd2ae53c8e93c1ba2c6e4865ab27"}
Nov 04 07:38:33.802881 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:33.802838    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" podStartSLOduration=2.8028230450000002 podStartE2EDuration="2.802823045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:33.799341402 +0000 UTC m=+1335.821556676" watchObservedRunningTime="2026-11-04 07:38:33.802823045 +0000 UTC m=+1335.825038305"
Nov 04 07:38:34.749309 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:34.749275    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:34.757051 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:34.757023    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:34.786766 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:34.786735    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:34.787867 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:34.787843    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:38:36.174088 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:38:36.174054    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:38:36.803037 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:36.803000    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8" event={"ID":"8c968ff1-4226-4571-8726-0c7f2f463e11","Type":"ContainerStarted","Data":"746a86e8ca93865652c5359f11d1357e1c84c4e8a7edb733f7b1789cb1177ab3"}
Nov 04 07:38:36.828279 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:36.828231    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8" podStartSLOduration=2.334868936 podStartE2EDuration="4.828218153s" totalImagesPullingTime="2.493349217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:36.825006537 +0000 UTC m=+1338.847221797" watchObservedRunningTime="2026-11-04 07:38:36.828218153 +0000 UTC m=+1338.850433413"
Nov 04 07:38:37.537571 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.537531    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"]
Nov 04 07:38:37.541940 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.541919    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"
Nov 04 07:38:37.561576 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.561550    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-5c1e58\""
Nov 04 07:38:37.561726 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.561707    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-5c1e58-dockercfg-4vgcs\""
Nov 04 07:38:37.569872 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.569850    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"]
Nov 04 07:38:37.612717 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.612684    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"]
Nov 04 07:38:37.617090 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.617069    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"
Nov 04 07:38:37.646650 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.646622    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"]
Nov 04 07:38:37.670096 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.670062    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn\" (UID: \"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"
Nov 04 07:38:37.670252 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.670228    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xh88\" (UniqueName: \"kubernetes.io/projected/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-kube-api-access-8xh88\") pod \"payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg\" (UID: \"cd960e9e-0f1d-44f8-9256-92da85ab0ac7\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"
Nov 04 07:38:37.670311 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.670295    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9pb4\" (UniqueName: \"kubernetes.io/projected/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-kube-api-access-r9pb4\") pod \"payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn\" (UID: \"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"
Nov 04 07:38:37.670398 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.670317    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg\" (UID: \"cd960e9e-0f1d-44f8-9256-92da85ab0ac7\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"
Nov 04 07:38:37.771839 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.771795    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xh88\" (UniqueName: \"kubernetes.io/projected/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-kube-api-access-8xh88\") pod \"payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg\" (UID: \"cd960e9e-0f1d-44f8-9256-92da85ab0ac7\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"
Nov 04 07:38:37.772008 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.771904    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9pb4\" (UniqueName: \"kubernetes.io/projected/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-kube-api-access-r9pb4\") pod \"payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn\" (UID: \"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"
Nov 04 07:38:37.772008 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.771932    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg\" (UID: \"cd960e9e-0f1d-44f8-9256-92da85ab0ac7\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"
Nov 04 07:38:37.772125 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.772108    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn\" (UID: \"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"
Nov 04 07:38:37.772807 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.772780    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg\" (UID: \"cd960e9e-0f1d-44f8-9256-92da85ab0ac7\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"
Nov 04 07:38:37.772916 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.772814    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn\" (UID: \"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"
Nov 04 07:38:37.785687 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.785658    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xh88\" (UniqueName: \"kubernetes.io/projected/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-kube-api-access-8xh88\") pod \"payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg\" (UID: \"cd960e9e-0f1d-44f8-9256-92da85ab0ac7\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"
Nov 04 07:38:37.786257 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.786238    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9pb4\" (UniqueName: \"kubernetes.io/projected/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-kube-api-access-r9pb4\") pod \"payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn\" (UID: \"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"
Nov 04 07:38:37.813124 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.813062    5626 generic.go:356] "Generic (PLEG): container finished" podID="8c968ff1-4226-4571-8726-0c7f2f463e11" containerID="746a86e8ca93865652c5359f11d1357e1c84c4e8a7edb733f7b1789cb1177ab3" exitCode=0
Nov 04 07:38:37.813124 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.813111    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8" event={"ID":"8c968ff1-4226-4571-8726-0c7f2f463e11","Type":"ContainerDied","Data":"746a86e8ca93865652c5359f11d1357e1c84c4e8a7edb733f7b1789cb1177ab3"}
Nov 04 07:38:37.879646 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.879603    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"
Nov 04 07:38:37.931573 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:37.931303    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"
Nov 04 07:38:38.283789 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:38.283756    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"]
Nov 04 07:38:38.287430 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:38:38.287402    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9ef5cbc_8ec6_4414_a74e_afe5c784b0dd.slice/crio-d6c886ac1966a399056ec38a806f51a471c48fc820e25848eb9e9ec4c100c9e8 WatchSource:0}: Error finding container d6c886ac1966a399056ec38a806f51a471c48fc820e25848eb9e9ec4c100c9e8: Status 404 returned error can't find the container with id d6c886ac1966a399056ec38a806f51a471c48fc820e25848eb9e9ec4c100c9e8
Nov 04 07:38:38.322349 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:38.322320    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"]
Nov 04 07:38:38.324886 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:38:38.324862    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd960e9e_0f1d_44f8_9256_92da85ab0ac7.slice/crio-07777870d119c4f812e6f003350b1b0b7906559a0e43e63091eb971f1e1dba9a WatchSource:0}: Error finding container 07777870d119c4f812e6f003350b1b0b7906559a0e43e63091eb971f1e1dba9a: Status 404 returned error can't find the container with id 07777870d119c4f812e6f003350b1b0b7906559a0e43e63091eb971f1e1dba9a
Nov 04 07:38:38.825535 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:38.825491    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg" event={"ID":"cd960e9e-0f1d-44f8-9256-92da85ab0ac7","Type":"ContainerStarted","Data":"6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801"}
Nov 04 07:38:38.826027 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:38.825584    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg" event={"ID":"cd960e9e-0f1d-44f8-9256-92da85ab0ac7","Type":"ContainerStarted","Data":"07777870d119c4f812e6f003350b1b0b7906559a0e43e63091eb971f1e1dba9a"}
Nov 04 07:38:38.826027 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:38.825752    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"
Nov 04 07:38:38.827424 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:38.827390    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn" event={"ID":"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd","Type":"ContainerStarted","Data":"ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5"}
Nov 04 07:38:38.827596 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:38.827450    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn" event={"ID":"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd","Type":"ContainerStarted","Data":"d6c886ac1966a399056ec38a806f51a471c48fc820e25848eb9e9ec4c100c9e8"}
Nov 04 07:38:38.827655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:38.827616    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"
Nov 04 07:38:38.852712 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:38.852656    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg" podStartSLOduration=1.852640547 podStartE2EDuration="1.852640547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:38.846932099 +0000 UTC m=+1340.869147363" watchObservedRunningTime="2026-11-04 07:38:38.852640547 +0000 UTC m=+1340.874855806"
Nov 04 07:38:38.884005 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:38.883937    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn" podStartSLOduration=1.88391832 podStartE2EDuration="1.88391832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:38.88367077 +0000 UTC m=+1340.905886030" watchObservedRunningTime="2026-11-04 07:38:38.88391832 +0000 UTC m=+1340.906133581"
Nov 04 07:38:39.009608 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.009556    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:39.111472 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.111346    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-openshift-ca\") pod \"8c968ff1-4226-4571-8726-0c7f2f463e11\" (UID: \"8c968ff1-4226-4571-8726-0c7f2f463e11\") "
Nov 04 07:38:39.111632 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.111470    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-xks-ca\") pod \"8c968ff1-4226-4571-8726-0c7f2f463e11\" (UID: \"8c968ff1-4226-4571-8726-0c7f2f463e11\") "
Nov 04 07:38:39.111678 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.111649    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c968ff1-4226-4571-8726-0c7f2f463e11-kube-api-access-h4hlg\" (UniqueName: \"kubernetes.io/projected/8c968ff1-4226-4571-8726-0c7f2f463e11-kube-api-access-h4hlg\") pod \"8c968ff1-4226-4571-8726-0c7f2f463e11\" (UID: \"8c968ff1-4226-4571-8726-0c7f2f463e11\") "
Nov 04 07:38:39.112263 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.112231    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-openshift-ca" pod "8c968ff1-4226-4571-8726-0c7f2f463e11" (UID: "8c968ff1-4226-4571-8726-0c7f2f463e11"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:38:39.115963 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.115931    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c968ff1-4226-4571-8726-0c7f2f463e11-kube-api-access-h4hlg" pod "8c968ff1-4226-4571-8726-0c7f2f463e11" (UID: "8c968ff1-4226-4571-8726-0c7f2f463e11"). InnerVolumeSpecName "kube-api-access-h4hlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:38:39.117219 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.117158    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-xks-ca" pod "8c968ff1-4226-4571-8726-0c7f2f463e11" (UID: "8c968ff1-4226-4571-8726-0c7f2f463e11"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:38:39.213859 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.213812    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:38:39.214036 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.213940    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4hlg\" (UniqueName: \"kubernetes.io/projected/8c968ff1-4226-4571-8726-0c7f2f463e11-kube-api-access-h4hlg\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:38:39.214036 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.213962    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c968ff1-4226-4571-8726-0c7f2f463e11-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:38:39.251240 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.251198    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"]
Nov 04 07:38:39.252166 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.252145    5626 cpu_manager.go:414] "containerMap: removing container" podUID="8c968ff1-4226-4571-8726-0c7f2f463e11" containerName="revoke-keys"
Nov 04 07:38:39.252243 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.252167    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c968ff1-4226-4571-8726-0c7f2f463e11" containerName="revoke-keys"
Nov 04 07:38:39.252297 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.252288    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c968ff1-4226-4571-8726-0c7f2f463e11" containerName="revoke-keys"
Nov 04 07:38:39.262836 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.262806    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.269456 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.269428    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"]
Nov 04 07:38:39.289224 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.289186    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-be2a74\"/\"payload-processing-external-model-e2e--30bfb805-dockercfg-x6rl9\""
Nov 04 07:38:39.289495 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.289205    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-be2a74\"/\"agc-cred-51aefd3f6d6a\""
Nov 04 07:38:39.292155 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.292129    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-be2a74\"/\"routing-overlay\""
Nov 04 07:38:39.292309 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.292277    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-be2a74\"/\"openshift-service-ca.crt\""
Nov 04 07:38:39.292423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.292404    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-be2a74\"/\"payload-processing-external-model-plugins-e2e-extmod-b-be2a74\""
Nov 04 07:38:39.292567 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.292552    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-be2a74\"/\"kube-root-ca.crt\""
Nov 04 07:38:39.316778 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.316183    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") " pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.316778 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.316321    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") " pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.316778 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.316459    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") " pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.316778 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.316486    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkmvc\" (UniqueName: \"kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-kube-api-access-pkmvc\") pod \"payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") " pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.426537 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.426185    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") " pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.426537 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.426248    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkmvc\" (UniqueName: \"kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-kube-api-access-pkmvc\") pod \"payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") " pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.426537 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.426471    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") " pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.426836 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.426554    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") " pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.427911 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.427881    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") " pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.428070 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.427955    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") " pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.430213 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.430148    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") " pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.444776 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.444743    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkmvc\" (UniqueName: \"kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-kube-api-access-pkmvc\") pod \"payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") " pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.579273 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.579230    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:39.836280 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.836239    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8" event={"ID":"8c968ff1-4226-4571-8726-0c7f2f463e11","Type":"ContainerDied","Data":"b545bff38e516807535712f60803da224bf51f31ca29d3ab5c2e935c72a802fd"}
Nov 04 07:38:39.836280 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.836267    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"
Nov 04 07:38:39.836851 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.836302    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b545bff38e516807535712f60803da224bf51f31ca29d3ab5c2e935c72a802fd"
Nov 04 07:38:39.867416 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:39.867382    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"]
Nov 04 07:38:39.868173 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:38:39.868141    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod316c48f5_de99_4977_a9a0_1ae5070ba20d.slice/crio-70cb0f1cdb873dad4807749de6b935cd4494279fafe1155e65ef6fa016fdd8f1 WatchSource:0}: Error finding container 70cb0f1cdb873dad4807749de6b935cd4494279fafe1155e65ef6fa016fdd8f1: Status 404 returned error can't find the container with id 70cb0f1cdb873dad4807749de6b935cd4494279fafe1155e65ef6fa016fdd8f1
Nov 04 07:38:40.846135 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:40.846090    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x" event={"ID":"316c48f5-de99-4977-a9a0-1ae5070ba20d","Type":"ContainerStarted","Data":"52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183"}
Nov 04 07:38:40.846541 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:40.846153    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x" event={"ID":"316c48f5-de99-4977-a9a0-1ae5070ba20d","Type":"ContainerStarted","Data":"70cb0f1cdb873dad4807749de6b935cd4494279fafe1155e65ef6fa016fdd8f1"}
Nov 04 07:38:40.846541 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:40.846182    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:40.885319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:40.885242    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x" podStartSLOduration=1.88522745 podStartE2EDuration="1.88522745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:40.884392217 +0000 UTC m=+1342.906607475" watchObservedRunningTime="2026-11-04 07:38:40.88522745 +0000 UTC m=+1342.907442756"
Nov 04 07:38:46.036824 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.036786    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"]
Nov 04 07:38:46.069663 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.069631    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"]
Nov 04 07:38:46.070028 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.069960    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:46.076183 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.076156    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:38:46.076294 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.076184    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:38:46.076408 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.076311    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:38:46.076408 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.076325    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:38:46.078612 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.078587    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhxp6\" (UniqueName: \"kubernetes.io/projected/f5aed379-b54c-4513-a0e6-20973954e19a-kube-api-access-zhxp6\") pod \"maas-api-revoke-sub-39d3b012-4f66t\" (UID: \"f5aed379-b54c-4513-a0e6-20973954e19a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:46.078719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.078630    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-39d3b012-4f66t\" (UID: \"f5aed379-b54c-4513-a0e6-20973954e19a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:46.078862 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.078845    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-39d3b012-4f66t\" (UID: \"f5aed379-b54c-4513-a0e6-20973954e19a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:46.180557 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.180520    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-39d3b012-4f66t\" (UID: \"f5aed379-b54c-4513-a0e6-20973954e19a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:46.180743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.180662    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhxp6\" (UniqueName: \"kubernetes.io/projected/f5aed379-b54c-4513-a0e6-20973954e19a-kube-api-access-zhxp6\") pod \"maas-api-revoke-sub-39d3b012-4f66t\" (UID: \"f5aed379-b54c-4513-a0e6-20973954e19a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:46.180743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.180718    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-39d3b012-4f66t\" (UID: \"f5aed379-b54c-4513-a0e6-20973954e19a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:46.181440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.181412    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-39d3b012-4f66t\" (UID: \"f5aed379-b54c-4513-a0e6-20973954e19a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:46.183095 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.183074    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-39d3b012-4f66t\" (UID: \"f5aed379-b54c-4513-a0e6-20973954e19a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:46.195964 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.195930    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhxp6\" (UniqueName: \"kubernetes.io/projected/f5aed379-b54c-4513-a0e6-20973954e19a-kube-api-access-zhxp6\") pod \"maas-api-revoke-sub-39d3b012-4f66t\" (UID: \"f5aed379-b54c-4513-a0e6-20973954e19a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:46.387568 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.387491    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:46.698396 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.698353    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"]
Nov 04 07:38:46.699315 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:38:46.699289    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5aed379_b54c_4513_a0e6_20973954e19a.slice/crio-d9943de98806a3ff8f1f9989fb588261a87a19e17696cafba40fc2c2ec043135 WatchSource:0}: Error finding container d9943de98806a3ff8f1f9989fb588261a87a19e17696cafba40fc2c2ec043135: Status 404 returned error can't find the container with id d9943de98806a3ff8f1f9989fb588261a87a19e17696cafba40fc2c2ec043135
Nov 04 07:38:46.910539 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:46.910506    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t" event={"ID":"f5aed379-b54c-4513-a0e6-20973954e19a","Type":"ContainerStarted","Data":"d9943de98806a3ff8f1f9989fb588261a87a19e17696cafba40fc2c2ec043135"}
Nov 04 07:38:47.919628 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:47.919593    5626 generic.go:356] "Generic (PLEG): container finished" podID="f5aed379-b54c-4513-a0e6-20973954e19a" containerID="e6e7d344ecad85bc178aacda98a67ab09dea4c344c0154ac84ecf5121217e9ca" exitCode=0
Nov 04 07:38:47.920010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:47.919631    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t" event={"ID":"f5aed379-b54c-4513-a0e6-20973954e19a","Type":"ContainerDied","Data":"e6e7d344ecad85bc178aacda98a67ab09dea4c344c0154ac84ecf5121217e9ca"}
Nov 04 07:38:49.131132 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.131105    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:49.224549 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.224451    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-openshift-ca\") pod \"f5aed379-b54c-4513-a0e6-20973954e19a\" (UID: \"f5aed379-b54c-4513-a0e6-20973954e19a\") "
Nov 04 07:38:49.224733 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.224659    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5aed379-b54c-4513-a0e6-20973954e19a-kube-api-access-zhxp6\" (UniqueName: \"kubernetes.io/projected/f5aed379-b54c-4513-a0e6-20973954e19a-kube-api-access-zhxp6\") pod \"f5aed379-b54c-4513-a0e6-20973954e19a\" (UID: \"f5aed379-b54c-4513-a0e6-20973954e19a\") "
Nov 04 07:38:49.224902 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.224881    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-xks-ca\") pod \"f5aed379-b54c-4513-a0e6-20973954e19a\" (UID: \"f5aed379-b54c-4513-a0e6-20973954e19a\") "
Nov 04 07:38:49.224902 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.224881    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-openshift-ca" pod "f5aed379-b54c-4513-a0e6-20973954e19a" (UID: "f5aed379-b54c-4513-a0e6-20973954e19a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:38:49.225428 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.225403    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:38:49.227207 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.227166    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-xks-ca" pod "f5aed379-b54c-4513-a0e6-20973954e19a" (UID: "f5aed379-b54c-4513-a0e6-20973954e19a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:38:49.228320 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.228289    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5aed379-b54c-4513-a0e6-20973954e19a-kube-api-access-zhxp6" pod "f5aed379-b54c-4513-a0e6-20973954e19a" (UID: "f5aed379-b54c-4513-a0e6-20973954e19a"). InnerVolumeSpecName "kube-api-access-zhxp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:38:49.329614 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.327808    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhxp6\" (UniqueName: \"kubernetes.io/projected/f5aed379-b54c-4513-a0e6-20973954e19a-kube-api-access-zhxp6\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:38:49.329614 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.327937    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5aed379-b54c-4513-a0e6-20973954e19a-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:38:49.839419 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.839391    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"
Nov 04 07:38:49.839561 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.839463    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"
Nov 04 07:38:49.938211 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.938179    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"
Nov 04 07:38:49.938211 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.938204    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t" event={"ID":"f5aed379-b54c-4513-a0e6-20973954e19a","Type":"ContainerDied","Data":"d9943de98806a3ff8f1f9989fb588261a87a19e17696cafba40fc2c2ec043135"}
Nov 04 07:38:49.938445 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:49.938288    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9943de98806a3ff8f1f9989fb588261a87a19e17696cafba40fc2c2ec043135"
Nov 04 07:38:50.597471 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:50.597441    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"]
Nov 04 07:38:51.866016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:51.865986    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:38:53.973269 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:53.973235    5626 generic.go:356] "Generic (PLEG): container finished" podID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerID="1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8" exitCode=0
Nov 04 07:38:53.973746 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:53.973304    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" event={"ID":"c915aa8b-a06f-4446-a012-dd9c0f383b7f","Type":"ContainerDied","Data":"1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8"}
Nov 04 07:38:53.973826 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:53.973801    5626 scope.go:122] "RemoveContainer" containerID="1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8"
Nov 04 07:38:53.975432 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:53.975401    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:53.975501 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:53.975470    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:53.975544 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:53.975502    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:38:54.982877 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:54.982839    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" event={"ID":"c915aa8b-a06f-4446-a012-dd9c0f383b7f","Type":"ContainerStarted","Data":"e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350"}
Nov 04 07:38:55.562389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:55.556888    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:55.562389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:55.561488    5626 patch_prober.go:29] interesting pod/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 07:38:55.562389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:55.561548    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 07:38:56.265046 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.265011    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"]
Nov 04 07:38:56.265767 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.265751    5626 cpu_manager.go:414] "containerMap: removing container" podUID="f5aed379-b54c-4513-a0e6-20973954e19a" containerName="revoke-keys"
Nov 04 07:38:56.265815 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.265768    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5aed379-b54c-4513-a0e6-20973954e19a" containerName="revoke-keys"
Nov 04 07:38:56.265918 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.265908    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5aed379-b54c-4513-a0e6-20973954e19a" containerName="revoke-keys"
Nov 04 07:38:56.273423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.273385    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:38:56.278352 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.278320    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:38:56.278530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.278350    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:38:56.278530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.278466    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"]
Nov 04 07:38:56.300298 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.300231    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:38:56.300499 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.300356    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:38:56.329721 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.329680    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x679p\" (UniqueName: \"kubernetes.io/projected/31ee0f07-60d1-4aaf-a327-feafad46276f-kube-api-access-x679p\") pod \"maas-api-revoke-sub-2a2a6a3e-8wh97\" (UID: \"31ee0f07-60d1-4aaf-a327-feafad46276f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:38:56.329900 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.329805    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2a2a6a3e-8wh97\" (UID: \"31ee0f07-60d1-4aaf-a327-feafad46276f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:38:56.329979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.329963    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2a2a6a3e-8wh97\" (UID: \"31ee0f07-60d1-4aaf-a327-feafad46276f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:38:56.431081 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.431039    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2a2a6a3e-8wh97\" (UID: \"31ee0f07-60d1-4aaf-a327-feafad46276f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:38:56.431288 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.431218    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2a2a6a3e-8wh97\" (UID: \"31ee0f07-60d1-4aaf-a327-feafad46276f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:38:56.431390 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.431339    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x679p\" (UniqueName: \"kubernetes.io/projected/31ee0f07-60d1-4aaf-a327-feafad46276f-kube-api-access-x679p\") pod \"maas-api-revoke-sub-2a2a6a3e-8wh97\" (UID: \"31ee0f07-60d1-4aaf-a327-feafad46276f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:38:56.431800 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.431781    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2a2a6a3e-8wh97\" (UID: \"31ee0f07-60d1-4aaf-a327-feafad46276f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:38:56.433754 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.433730    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2a2a6a3e-8wh97\" (UID: \"31ee0f07-60d1-4aaf-a327-feafad46276f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:38:56.464539 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.464504    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x679p\" (UniqueName: \"kubernetes.io/projected/31ee0f07-60d1-4aaf-a327-feafad46276f-kube-api-access-x679p\") pod \"maas-api-revoke-sub-2a2a6a3e-8wh97\" (UID: \"31ee0f07-60d1-4aaf-a327-feafad46276f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:38:56.558388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.556804    5626 patch_prober.go:29] interesting pod/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 07:38:56.558388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.556869    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 07:38:56.590984 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.590954    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:38:56.895607 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:56.895576    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"]
Nov 04 07:38:56.898003 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:38:56.897975    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ee0f07_60d1_4aaf_a327_feafad46276f.slice/crio-3ea974a35399fba1b8676771efc35b0df4f831803e27a5984c1a2e479e8dd1d3 WatchSource:0}: Error finding container 3ea974a35399fba1b8676771efc35b0df4f831803e27a5984c1a2e479e8dd1d3: Status 404 returned error can't find the container with id 3ea974a35399fba1b8676771efc35b0df4f831803e27a5984c1a2e479e8dd1d3
Nov 04 07:38:57.004856 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:57.004820    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97" event={"ID":"31ee0f07-60d1-4aaf-a327-feafad46276f","Type":"ContainerStarted","Data":"3ea974a35399fba1b8676771efc35b0df4f831803e27a5984c1a2e479e8dd1d3"}
Nov 04 07:38:57.562386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:57.561790    5626 patch_prober.go:29] interesting pod/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 07:38:57.562386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:57.561853    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 07:38:58.015120 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:58.009490    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"]
Nov 04 07:38:58.038420 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:58.030098    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97" event={"ID":"31ee0f07-60d1-4aaf-a327-feafad46276f","Type":"ContainerStarted","Data":"2bda2356582019b15f592a9263887d38c5b0590c6b11eb116a21b3d0d3b29c9b"}
Nov 04 07:38:58.038420 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:58.035022    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"]
Nov 04 07:38:58.129418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:58.129304    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97" podStartSLOduration=2.129284749 podStartE2EDuration="2.129284749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:58.121803904 +0000 UTC m=+1360.144019164" watchObservedRunningTime="2026-11-04 07:38:58.129284749 +0000 UTC m=+1360.151500010"
Nov 04 07:38:58.263543 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:58.261477    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:38:58.567388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:58.555983    5626 patch_prober.go:29] interesting pod/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 07:38:58.567388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:58.556074    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 07:38:59.076035 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:59.075356    5626 generic.go:356] "Generic (PLEG): container finished" podID="31ee0f07-60d1-4aaf-a327-feafad46276f" containerID="2bda2356582019b15f592a9263887d38c5b0590c6b11eb116a21b3d0d3b29c9b" exitCode=0
Nov 04 07:38:59.076035 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:59.075593    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97" event={"ID":"31ee0f07-60d1-4aaf-a327-feafad46276f","Type":"ContainerDied","Data":"2bda2356582019b15f592a9263887d38c5b0590c6b11eb116a21b3d0d3b29c9b"}
Nov 04 07:38:59.566392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:59.564869    5626 patch_prober.go:29] interesting pod/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 07:38:59.566392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:38:59.564976    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 07:39:00.592388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:00.585121    5626 patch_prober.go:29] interesting pod/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 07:39:00.592388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:00.585192    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 07:39:00.825389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:00.825047    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:01.184388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.183541    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 07:39:01.209779 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.184748    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:01.520899 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.515666    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:01.561965 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.561487    5626 patch_prober.go:29] interesting pod/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 04 07:39:01.561965 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.561566    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 04 07:39:01.604397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.591049    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:39:01.713434 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.710058    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31ee0f07-60d1-4aaf-a327-feafad46276f-kube-api-access-x679p\" (UniqueName: \"kubernetes.io/projected/31ee0f07-60d1-4aaf-a327-feafad46276f-kube-api-access-x679p\") pod \"31ee0f07-60d1-4aaf-a327-feafad46276f\" (UID: \"31ee0f07-60d1-4aaf-a327-feafad46276f\") "
Nov 04 07:39:01.724425 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.713813    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-openshift-ca\") pod \"31ee0f07-60d1-4aaf-a327-feafad46276f\" (UID: \"31ee0f07-60d1-4aaf-a327-feafad46276f\") "
Nov 04 07:39:01.725006 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.724973    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-xks-ca\") pod \"31ee0f07-60d1-4aaf-a327-feafad46276f\" (UID: \"31ee0f07-60d1-4aaf-a327-feafad46276f\") "
Nov 04 07:39:01.740811 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.740769    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-xgjgh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:39:01.741129 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.741095    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh" podUID="7ba40135-db97-4238-8bb8-f1fd581a1615" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:01.741913 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.741878    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-openshift-ca" pod "31ee0f07-60d1-4aaf-a327-feafad46276f" (UID: "31ee0f07-60d1-4aaf-a327-feafad46276f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:39:01.806294 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.806240    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 04 07:39:01.807491 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.807457    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Nov 04 07:39:01.827152 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.827110    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:01.828779 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.828355    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 04 07:39:01.835536 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:01.835507    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:39:02.056016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.055675    5626 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:02.076788 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.076552    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-xks-ca" pod "31ee0f07-60d1-4aaf-a327-feafad46276f" (UID: "31ee0f07-60d1-4aaf-a327-feafad46276f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:39:02.099791 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.099581    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ee0f07-60d1-4aaf-a327-feafad46276f-kube-api-access-x679p" pod "31ee0f07-60d1-4aaf-a327-feafad46276f" (UID: "31ee0f07-60d1-4aaf-a327-feafad46276f"). InnerVolumeSpecName "kube-api-access-x679p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:02.193922 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.175956    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31ee0f07-60d1-4aaf-a327-feafad46276f-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:02.193922 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.175995    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x679p\" (UniqueName: \"kubernetes.io/projected/31ee0f07-60d1-4aaf-a327-feafad46276f-kube-api-access-x679p\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:02.193922 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.192894    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:02.290390 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.250276    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:39:02.737424 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.727159    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:02.787016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.786972    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97" event={"ID":"31ee0f07-60d1-4aaf-a327-feafad46276f","Type":"ContainerDied","Data":"3ea974a35399fba1b8676771efc35b0df4f831803e27a5984c1a2e479e8dd1d3"}
Nov 04 07:39:02.787220 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.787055    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ea974a35399fba1b8676771efc35b0df4f831803e27a5984c1a2e479e8dd1d3"
Nov 04 07:39:02.787220 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.787157    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"
Nov 04 07:39:02.823389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.812552    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Nov 04 07:39:02.823994 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.822519    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:02.824385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.824336    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:39:02.824501 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.824427    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:39:02.846873 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:02.846827    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:03.048298 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.038026    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k" podUID="a8b6a0dc-8600-4ab9-923f-3d680473f39f" containerName="payload-processing" containerID="cri-o://65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2" gracePeriod=25
Nov 04 07:39:03.048298 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.038254    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt" podUID="9b8fb283-b77c-4758-9497-84ba24890da2" containerName="payload-pre-processing" containerID="cri-o://0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce" gracePeriod=25
Nov 04 07:39:03.234471 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.194490    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:03.234471 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.194882    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 07:39:03.234471 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.234107    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" podUID="573aaebc-ca66-4bf2-84a5-2057a5e77b87" containerName="main" probeResult="failure" output="Get \"https://10.134.0.38:8000/ready\": context deadline exceeded"
Nov 04 07:39:03.414446 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.413728    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:39:03.433737 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.433621    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:39:03.516383 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.515730    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:03.759608 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.745870    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:03.765386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.761713    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:39:03.830761 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.830710    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:03.841010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.840965    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Nov 04 07:39:03.953551 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.953482    5626 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": context deadline exceeded"
Nov 04 07:39:04.032460 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:03.954892    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 04 07:39:04.233386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:04.220879    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:04.233386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:04.221264    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 04 07:39:04.246560 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:04.234996    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" podUID="573aaebc-ca66-4bf2-84a5-2057a5e77b87" containerName="main" probeResult="failure" output="Get \"https://10.134.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:04.349865 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:04.349761    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:39:04.350148 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:04.350130    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:39:04.621445 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:04.611113    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:04.675386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:04.623164    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:39:04.794334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:04.785794    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:04.873623 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:04.873111    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:04.885555 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:04.885507    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:04.911389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:04.893587    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:39:04.944997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:04.944964    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:39:05.048467 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.041496    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:39:05.048467 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.041559    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:05.147446 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.113084    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:39:05.147446 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.113147    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:05.224998 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.224943    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:05.226394 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.225262    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:39:05.249385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.243102    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:39:05.323424 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.322427    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:39:05.442447 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.441142    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:39:05.442447 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.442233    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:39:05.622034 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.621989    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:39:05.633152 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.633125    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:39:05.739443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.730954    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" containerID="cri-o://15d59189554b3b3b0439abde9e9cef6fd6863c3f08e0fe4523db58985296b7d9" gracePeriod=45
Nov 04 07:39:05.869210 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:05.830597    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": read tcp 10.134.0.2:59550->10.134.0.60:8000: read: connection reset by peer"
Nov 04 07:39:06.336983 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:39:06.336938    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:39:06.348621 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.348587    5626 generic.go:356] "Generic (PLEG): container finished" podID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerID="15d59189554b3b3b0439abde9e9cef6fd6863c3f08e0fe4523db58985296b7d9" exitCode=0
Nov 04 07:39:06.349079 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.349058    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" event={"ID":"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7","Type":"ContainerDied","Data":"15d59189554b3b3b0439abde9e9cef6fd6863c3f08e0fe4523db58985296b7d9"}
Nov 04 07:39:06.510877 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.509472    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k" podUID="a8b6a0dc-8600-4ab9-923f-3d680473f39f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:39:06.535651 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.532506    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:39:06.590764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.587916    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tmp-dir\") pod \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") "
Nov 04 07:39:06.590764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.588054    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-model-cache\") pod \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") "
Nov 04 07:39:06.590764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.588128    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tls-certs\" (UniqueName: \"kubernetes.io/secret/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tls-certs\") pod \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") "
Nov 04 07:39:06.590764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.588163    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-kube-api-access-j7dz8\" (UniqueName: \"kubernetes.io/projected/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-kube-api-access-j7dz8\") pod \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") "
Nov 04 07:39:06.590764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.588200    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-home\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-home\") pod \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") "
Nov 04 07:39:06.590764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.588297    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-dshm\") pod \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\" (UID: \"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7\") "
Nov 04 07:39:06.590764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.590322    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tmp-dir" pod "efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" (UID: "efb9db9c-e485-4ad4-9ce8-e5fa791a11d7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:06.590764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.590534    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-model-cache" pod "efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" (UID: "efb9db9c-e485-4ad4-9ce8-e5fa791a11d7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:06.594493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.593247    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-home" pod "efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" (UID: "efb9db9c-e485-4ad4-9ce8-e5fa791a11d7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:06.638606 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.638554    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-kube-api-access-j7dz8" pod "efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" (UID: "efb9db9c-e485-4ad4-9ce8-e5fa791a11d7"). InnerVolumeSpecName "kube-api-access-j7dz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:06.692040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.686820    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-dshm" pod "efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" (UID: "efb9db9c-e485-4ad4-9ce8-e5fa791a11d7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:06.692040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.691061    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tls-certs" pod "efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" (UID: "efb9db9c-e485-4ad4-9ce8-e5fa791a11d7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:39:06.692040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.691764    5626 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-model-cache\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:06.692040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.691791    5626 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tls-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:06.692040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.691807    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7dz8\" (UniqueName: \"kubernetes.io/projected/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-kube-api-access-j7dz8\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:06.692040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.691822    5626 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-home\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:06.692040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.691844    5626 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-dshm\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:06.692040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:06.691856    5626 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7-tmp-dir\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:07.373948 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:07.373918    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"
Nov 04 07:39:07.374411 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:07.373951    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" event={"ID":"efb9db9c-e485-4ad4-9ce8-e5fa791a11d7","Type":"ContainerDied","Data":"7b615441d6ee8c485c5df68e33ca41b0250ef4fbdd49fe944045540eecf49d42"}
Nov 04 07:39:07.374411 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:07.374004    5626 scope.go:122] "RemoveContainer" containerID="15d59189554b3b3b0439abde9e9cef6fd6863c3f08e0fe4523db58985296b7d9"
Nov 04 07:39:07.418481 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:07.418449    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"]
Nov 04 07:39:07.433573 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:07.433537    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x"]
Nov 04 07:39:07.509852 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:07.509801    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-w3-60ed4a-kserve-dd89c878b-68l8x" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 04 07:39:07.511552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:07.511522    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt" podUID="9b8fb283-b77c-4758-9497-84ba24890da2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:39:08.096633 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:08.096589    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" path="/var/lib/kubelet/pods/efb9db9c-e485-4ad4-9ce8-e5fa791a11d7/volumes"
Nov 04 07:39:12.439187 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:12.439150    5626 generic.go:356] "Generic (PLEG): container finished" podID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerID="2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087" exitCode=0
Nov 04 07:39:12.440131 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:12.439200    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" event={"ID":"222aacaf-0c37-4aad-91ad-5607a50820ce","Type":"ContainerDied","Data":"2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087"}
Nov 04 07:39:12.440131 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:12.440033    5626 scope.go:122] "RemoveContainer" containerID="2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087"
Nov 04 07:39:12.442684 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:12.442634    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:39:12.442824 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:12.442769    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:39:12.442900 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:12.442833    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:39:13.471138 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:13.471102    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" event={"ID":"222aacaf-0c37-4aad-91ad-5607a50820ce","Type":"ContainerStarted","Data":"c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af"}
Nov 04 07:39:13.762448 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:13.756952    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:39:13.762448 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:13.757093    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 07:39:13.762448 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:13.757142    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 07:39:14.366982 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.366936    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"]
Nov 04 07:39:14.368031 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.368007    5626 cpu_manager.go:414] "containerMap: removing container" podUID="31ee0f07-60d1-4aaf-a327-feafad46276f" containerName="revoke-keys"
Nov 04 07:39:14.368094 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.368035    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31ee0f07-60d1-4aaf-a327-feafad46276f" containerName="revoke-keys"
Nov 04 07:39:14.368094 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.368067    5626 cpu_manager.go:414] "containerMap: removing container" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main"
Nov 04 07:39:14.368094 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.368075    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main"
Nov 04 07:39:14.368242 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.368229    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="efb9db9c-e485-4ad4-9ce8-e5fa791a11d7" containerName="main"
Nov 04 07:39:14.368283 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.368259    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="31ee0f07-60d1-4aaf-a327-feafad46276f" containerName="revoke-keys"
Nov 04 07:39:14.370804 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.370779    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"
Nov 04 07:39:14.379723 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.378278    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-a558a6\""
Nov 04 07:39:14.381809 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.381781    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"]
Nov 04 07:39:14.382671 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.382645    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-a558a6-dockercfg-4wvgt\""
Nov 04 07:39:14.406021 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.405984    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79j96\" (UniqueName: \"kubernetes.io/projected/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-kube-api-access-79j96\") pod \"payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm\" (UID: \"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"
Nov 04 07:39:14.406242 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.406187    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm\" (UID: \"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"
Nov 04 07:39:14.508318 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.508277    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79j96\" (UniqueName: \"kubernetes.io/projected/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-kube-api-access-79j96\") pod \"payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm\" (UID: \"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"
Nov 04 07:39:14.509101 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.509075    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm\" (UID: \"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"
Nov 04 07:39:14.510284 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.510259    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm\" (UID: \"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"
Nov 04 07:39:14.517832 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.517806    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"]
Nov 04 07:39:14.522492 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.522467    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.656444 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.655813    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"]
Nov 04 07:39:14.661396 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.660530    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.661396 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.660625    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.661396 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.660670    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.661396 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.660784    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7tk8\" (UniqueName: \"kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-kube-api-access-w7tk8\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.661396 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.660839    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.661396 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.660912    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.661396 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.660950    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.661396 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.661011    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.661396 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.661045    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.687008 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.685179    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79j96\" (UniqueName: \"kubernetes.io/projected/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-kube-api-access-79j96\") pod \"payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm\" (UID: \"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"
Nov 04 07:39:14.687199 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.687128    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"]
Nov 04 07:39:14.687265 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.687246    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"
Nov 04 07:39:14.688807 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.688212    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"
Nov 04 07:39:14.691476 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.690794    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"]
Nov 04 07:39:14.695923 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.695888    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"]
Nov 04 07:39:14.700330 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.700285    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.705763 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.705736    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"]
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769042    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn\" (UID: \"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769097    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769156    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769195    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769223    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cca66d0d-442b-4138-99a3-72bc0579f077-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769271    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrp87\" (UniqueName: \"kubernetes.io/projected/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-kube-api-access-wrp87\") pod \"payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn\" (UID: \"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769306    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769339    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769427    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769502    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cca66d0d-442b-4138-99a3-72bc0579f077-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769559    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769652    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769715    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769742    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769778    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769808    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769855    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769916    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dq4z\" (UniqueName: \"kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-kube-api-access-2dq4z\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.769977    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7tk8\" (UniqueName: \"kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-kube-api-access-w7tk8\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.770418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.770040    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.771712 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.770672    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.771712 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.770948    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.771712 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.771328    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 07:39:14.771712 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.771390    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 07:39:14.772029 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.772003    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.772272 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.772240    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.772641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.772614    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.775420 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.775376    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.775975 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.775954    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.794352 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.794321    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.805818 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.805788    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7tk8\" (UniqueName: \"kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-kube-api-access-w7tk8\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.861518 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.859772    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:14.872789 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.872705    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cca66d0d-442b-4138-99a3-72bc0579f077-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.872789 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.872773    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.873308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.872925    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.873308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.872953    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.873308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.872985    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.873308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.873051    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dq4z\" (UniqueName: \"kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-kube-api-access-2dq4z\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.873308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.873150    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn\" (UID: \"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"
Nov 04 07:39:14.873308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.873172    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.873308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.873225    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cca66d0d-442b-4138-99a3-72bc0579f077-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.873308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.873267    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrp87\" (UniqueName: \"kubernetes.io/projected/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-kube-api-access-wrp87\") pod \"payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn\" (UID: \"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"
Nov 04 07:39:14.873308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.873308    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.874746 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.874708    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.875405 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.875353    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.877661 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.875805    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.877798 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.876572    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn\" (UID: \"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"
Nov 04 07:39:14.877896 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.877110    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cca66d0d-442b-4138-99a3-72bc0579f077-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.877982 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.877602    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.879227 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.879179    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cca66d0d-442b-4138-99a3-72bc0579f077-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.880906 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.880862    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.892416 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.887455    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:14.895745 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.895694    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrp87\" (UniqueName: \"kubernetes.io/projected/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-kube-api-access-wrp87\") pod \"payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn\" (UID: \"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"
Nov 04 07:39:14.896566 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:14.896532    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dq4z\" (UniqueName: \"kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-kube-api-access-2dq4z\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:15.026121 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.026039    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"]
Nov 04 07:39:15.029594 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:39:15.029563    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69fe62cf_c6f0_4fc5_83fb_151e33cfa0c3.slice/crio-fb37a507115eb63e81a8aa4ccb8115df827ac31f6b8a223b00bb682407153e27 WatchSource:0}: Error finding container fb37a507115eb63e81a8aa4ccb8115df827ac31f6b8a223b00bb682407153e27: Status 404 returned error can't find the container with id fb37a507115eb63e81a8aa4ccb8115df827ac31f6b8a223b00bb682407153e27
Nov 04 07:39:15.073846 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.073797    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"
Nov 04 07:39:15.125016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.124995    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:15.195656 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.195567    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"]
Nov 04 07:39:15.203505 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:39:15.203476    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod512d84d2_e30f_4c72_97da_cb58ccb2c8e5.slice/crio-4c51d2aa37cd53ec0f487519c608f5e5fda818afa353b9e2e79065c87ce9158c WatchSource:0}: Error finding container 4c51d2aa37cd53ec0f487519c608f5e5fda818afa353b9e2e79065c87ce9158c: Status 404 returned error can't find the container with id 4c51d2aa37cd53ec0f487519c608f5e5fda818afa353b9e2e79065c87ce9158c
Nov 04 07:39:15.206143 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.206103    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:39:15.206254 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.206196    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:39:15.206254 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.206242    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:39:15.401309 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.401268    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"]
Nov 04 07:39:15.404210 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:39:15.404181    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc77902_2fd9_47b8_b8f3_b95a2ecd0b7a.slice/crio-543b7eab241ceafb3e7ab5e00d4d6e4467ef6b1be1f76d3e0aa1aaeebd48ece1 WatchSource:0}: Error finding container 543b7eab241ceafb3e7ab5e00d4d6e4467ef6b1be1f76d3e0aa1aaeebd48ece1: Status 404 returned error can't find the container with id 543b7eab241ceafb3e7ab5e00d4d6e4467ef6b1be1f76d3e0aa1aaeebd48ece1
Nov 04 07:39:15.521452 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.521422    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"]
Nov 04 07:39:15.521968 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:39:15.521824    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcca66d0d_442b_4138_99a3_72bc0579f077.slice/crio-d3646f5f7ebfb78c83dae827cf72e341e8b8a8faf32231544fe455d59f6d06a9 WatchSource:0}: Error finding container d3646f5f7ebfb78c83dae827cf72e341e8b8a8faf32231544fe455d59f6d06a9: Status 404 returned error can't find the container with id d3646f5f7ebfb78c83dae827cf72e341e8b8a8faf32231544fe455d59f6d06a9
Nov 04 07:39:15.526046 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.525995    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:39:15.526148 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.526112    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:39:15.526226 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.526160    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:39:15.547016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.546975    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q" event={"ID":"512d84d2-e30f-4c72-97da-cb58ccb2c8e5","Type":"ContainerStarted","Data":"42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d"}
Nov 04 07:39:15.547326 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.547292    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q" event={"ID":"512d84d2-e30f-4c72-97da-cb58ccb2c8e5","Type":"ContainerStarted","Data":"4c51d2aa37cd53ec0f487519c608f5e5fda818afa353b9e2e79065c87ce9158c"}
Nov 04 07:39:15.583063 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.583024    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn" event={"ID":"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a","Type":"ContainerStarted","Data":"543b7eab241ceafb3e7ab5e00d4d6e4467ef6b1be1f76d3e0aa1aaeebd48ece1"}
Nov 04 07:39:15.590639 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.590582    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm" event={"ID":"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3","Type":"ContainerStarted","Data":"ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54"}
Nov 04 07:39:15.590972 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.590947    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm" event={"ID":"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3","Type":"ContainerStarted","Data":"fb37a507115eb63e81a8aa4ccb8115df827ac31f6b8a223b00bb682407153e27"}
Nov 04 07:39:15.594276 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.592292    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"
Nov 04 07:39:15.599734 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.599675    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q" podStartSLOduration=1.599656858 podStartE2EDuration="1.599656858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:39:15.589981951 +0000 UTC m=+1377.612197212" watchObservedRunningTime="2026-11-04 07:39:15.599656858 +0000 UTC m=+1377.621872119"
Nov 04 07:39:15.626050 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.625987    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm" podStartSLOduration=1.6259670160000002 podStartE2EDuration="1.625967016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:39:15.622606058 +0000 UTC m=+1377.644821330" watchObservedRunningTime="2026-11-04 07:39:15.625967016 +0000 UTC m=+1377.648182277"
Nov 04 07:39:15.751437 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.751323    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 07:39:15.751712 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.751669    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 07:39:15.869506 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:15.869468    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:16.515392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:16.515299    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k" podUID="a8b6a0dc-8600-4ab9-923f-3d680473f39f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:39:16.623978 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:16.623940    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9" event={"ID":"cca66d0d-442b-4138-99a3-72bc0579f077","Type":"ContainerStarted","Data":"7183bf97ffa76ae700dc3426594e2525d52f77ad1866d17819c990144998deaf"}
Nov 04 07:39:16.624482 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:16.624006    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9" event={"ID":"cca66d0d-442b-4138-99a3-72bc0579f077","Type":"ContainerStarted","Data":"d3646f5f7ebfb78c83dae827cf72e341e8b8a8faf32231544fe455d59f6d06a9"}
Nov 04 07:39:16.626120 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:16.626093    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn" event={"ID":"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a","Type":"ContainerStarted","Data":"f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3"}
Nov 04 07:39:16.626280 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:16.626270    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"
Nov 04 07:39:16.688389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:16.687419    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9" podStartSLOduration=2.687391953 podStartE2EDuration="2.687391953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:39:16.660986526 +0000 UTC m=+1378.683201810" watchObservedRunningTime="2026-11-04 07:39:16.687391953 +0000 UTC m=+1378.709607215"
Nov 04 07:39:16.719319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:16.719260    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn" podStartSLOduration=2.719243445 podStartE2EDuration="2.719243445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:39:16.702811766 +0000 UTC m=+1378.725027018" watchObservedRunningTime="2026-11-04 07:39:16.719243445 +0000 UTC m=+1378.741458705"
Nov 04 07:39:16.754387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:16.751815    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 07:39:16.754387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:16.751894    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 07:39:16.873310 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:16.873272    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 04 07:39:16.873631 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:16.873601    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q" podUID="512d84d2-e30f-4c72-97da-cb58ccb2c8e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 04 07:39:17.125634 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:17.125527    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:17.127292 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:17.127165    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 04 07:39:17.127525 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:17.127500    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9" podUID="cca66d0d-442b-4138-99a3-72bc0579f077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 04 07:39:17.518395 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:17.517807    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt" podUID="9b8fb283-b77c-4758-9497-84ba24890da2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:39:17.759399 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:17.758878    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 07:39:17.759399 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:17.758942    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 07:39:17.876408 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:17.875875    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 04 07:39:17.876408 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:17.875960    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q" podUID="512d84d2-e30f-4c72-97da-cb58ccb2c8e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 04 07:39:18.136443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:18.132848    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 04 07:39:18.136443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:18.132900    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9" podUID="cca66d0d-442b-4138-99a3-72bc0579f077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 04 07:39:18.758388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:18.752948    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 07:39:18.758388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:18.753028    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 07:39:18.871212 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:18.870745    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 04 07:39:18.871212 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:18.870812    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q" podUID="512d84d2-e30f-4c72-97da-cb58ccb2c8e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 04 07:39:19.125319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:19.125284    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 04 07:39:19.125567 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:19.125334    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9" podUID="cca66d0d-442b-4138-99a3-72bc0579f077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 04 07:39:19.768387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:19.767678    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:39:19.769394 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:19.769300    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:39:19.784385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:19.783654    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:39:19.901679 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:19.880113    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 04 07:39:19.901679 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:19.900647    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q" podUID="512d84d2-e30f-4c72-97da-cb58ccb2c8e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 04 07:39:20.125957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:20.125916    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 04 07:39:20.126181 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:20.125966    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9" podUID="cca66d0d-442b-4138-99a3-72bc0579f077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 04 07:39:20.871961 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:20.871897    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 04 07:39:20.872162 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:20.871979    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q" podUID="512d84d2-e30f-4c72-97da-cb58ccb2c8e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 04 07:39:21.125705 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:21.125626    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 04 07:39:21.126144 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:21.126119    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9" podUID="cca66d0d-442b-4138-99a3-72bc0579f077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 04 07:39:21.867185 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:21.867143    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 04 07:39:21.867513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:21.867477    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q" podUID="512d84d2-e30f-4c72-97da-cb58ccb2c8e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 04 07:39:22.126055 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:22.125970    5626 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 04 07:39:22.126055 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:22.126027    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9" podUID="cca66d0d-442b-4138-99a3-72bc0579f077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 04 07:39:23.292719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.292545    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:23.294224 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.294184    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:23.298433 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.297563    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:23.460119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.460010    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"
Nov 04 07:39:23.561313 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.561280    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b8fb283-b77c-4758-9497-84ba24890da2-kube-api-access-j79lg\" (UniqueName: \"kubernetes.io/projected/9b8fb283-b77c-4758-9497-84ba24890da2-kube-api-access-j79lg\") pod \"9b8fb283-b77c-4758-9497-84ba24890da2\" (UID: \"9b8fb283-b77c-4758-9497-84ba24890da2\") "
Nov 04 07:39:23.561938 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.561917    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b8fb283-b77c-4758-9497-84ba24890da2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8fb283-b77c-4758-9497-84ba24890da2-plugins-config-volume\") pod \"9b8fb283-b77c-4758-9497-84ba24890da2\" (UID: \"9b8fb283-b77c-4758-9497-84ba24890da2\") "
Nov 04 07:39:23.562848 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.562633    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b8fb283-b77c-4758-9497-84ba24890da2-plugins-config-volume" pod "9b8fb283-b77c-4758-9497-84ba24890da2" (UID: "9b8fb283-b77c-4758-9497-84ba24890da2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:39:23.563861 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.563825    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b8fb283-b77c-4758-9497-84ba24890da2-kube-api-access-j79lg" pod "9b8fb283-b77c-4758-9497-84ba24890da2" (UID: "9b8fb283-b77c-4758-9497-84ba24890da2"). InnerVolumeSpecName "kube-api-access-j79lg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:23.564395 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.564374    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8fb283-b77c-4758-9497-84ba24890da2-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:23.564525 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.564512    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j79lg\" (UniqueName: \"kubernetes.io/projected/9b8fb283-b77c-4758-9497-84ba24890da2-kube-api-access-j79lg\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:23.565587 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.565567    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"
Nov 04 07:39:23.632803 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.632748    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:23.633540 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.633296    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:23.634745 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.634720    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:23.665707 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.665674    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8b6a0dc-8600-4ab9-923f-3d680473f39f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b6a0dc-8600-4ab9-923f-3d680473f39f-plugins-config-volume\") pod \"a8b6a0dc-8600-4ab9-923f-3d680473f39f\" (UID: \"a8b6a0dc-8600-4ab9-923f-3d680473f39f\") "
Nov 04 07:39:23.665707 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.665712    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8b6a0dc-8600-4ab9-923f-3d680473f39f-kube-api-access-sc7cc\" (UniqueName: \"kubernetes.io/projected/a8b6a0dc-8600-4ab9-923f-3d680473f39f-kube-api-access-sc7cc\") pod \"a8b6a0dc-8600-4ab9-923f-3d680473f39f\" (UID: \"a8b6a0dc-8600-4ab9-923f-3d680473f39f\") "
Nov 04 07:39:23.666173 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.666142    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8b6a0dc-8600-4ab9-923f-3d680473f39f-plugins-config-volume" pod "a8b6a0dc-8600-4ab9-923f-3d680473f39f" (UID: "a8b6a0dc-8600-4ab9-923f-3d680473f39f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:39:23.666708 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.666466    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b6a0dc-8600-4ab9-923f-3d680473f39f-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:23.668112 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.668084    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b6a0dc-8600-4ab9-923f-3d680473f39f-kube-api-access-sc7cc" pod "a8b6a0dc-8600-4ab9-923f-3d680473f39f" (UID: "a8b6a0dc-8600-4ab9-923f-3d680473f39f"). InnerVolumeSpecName "kube-api-access-sc7cc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:23.725884 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.725793    5626 generic.go:356] "Generic (PLEG): container finished" podID="a8b6a0dc-8600-4ab9-923f-3d680473f39f" containerID="65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2" exitCode=0
Nov 04 07:39:23.725884 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.725841    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"
Nov 04 07:39:23.726127 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.725887    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k" event={"ID":"a8b6a0dc-8600-4ab9-923f-3d680473f39f","Type":"ContainerDied","Data":"65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2"}
Nov 04 07:39:23.726127 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.725969    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k" event={"ID":"a8b6a0dc-8600-4ab9-923f-3d680473f39f","Type":"ContainerDied","Data":"bb33752bae1be2f5017ca01162ce3a59ce76f7b6ee41cf944066c5a0b0338f06"}
Nov 04 07:39:23.726127 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.725986    5626 scope.go:122] "RemoveContainer" containerID="65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2"
Nov 04 07:39:23.728815 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.728783    5626 generic.go:356] "Generic (PLEG): container finished" podID="9b8fb283-b77c-4758-9497-84ba24890da2" containerID="0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce" exitCode=0
Nov 04 07:39:23.731653 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.729821    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"
Nov 04 07:39:23.736237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.736211    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt" event={"ID":"9b8fb283-b77c-4758-9497-84ba24890da2","Type":"ContainerDied","Data":"0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce"}
Nov 04 07:39:23.736392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.736270    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt" event={"ID":"9b8fb283-b77c-4758-9497-84ba24890da2","Type":"ContainerDied","Data":"5b35f678cf63ec3331170fb304116baeb8f5188f97be09f6b0afb8e879054f85"}
Nov 04 07:39:23.745443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.745295    5626 scope.go:122] "RemoveContainer" containerID="65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2"
Nov 04 07:39:23.746701 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:39:23.746679    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2\": container with ID starting with 65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2 not found: ID does not exist" containerID="65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2"
Nov 04 07:39:23.746780 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.746709    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2"} err="failed to get container status \"65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2\": rpc error: code = NotFound desc = could not find container \"65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2\": container with ID starting with 65d1c4fedea35a08aa0e70392a6db573640f635903c45b40fe7877c5e5e43fb2 not found: ID does not exist"
Nov 04 07:39:23.746780 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.746726    5626 scope.go:122] "RemoveContainer" containerID="0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce"
Nov 04 07:39:23.769422 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.769391    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sc7cc\" (UniqueName: \"kubernetes.io/projected/a8b6a0dc-8600-4ab9-923f-3d680473f39f-kube-api-access-sc7cc\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:23.777265 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.777229    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"]
Nov 04 07:39:23.786553 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.785670    5626 scope.go:122] "RemoveContainer" containerID="0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce"
Nov 04 07:39:23.789418 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:39:23.787024    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce\": container with ID starting with 0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce not found: ID does not exist" containerID="0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce"
Nov 04 07:39:23.789418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.787063    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce"} err="failed to get container status \"0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce\": rpc error: code = NotFound desc = could not find container \"0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce\": container with ID starting with 0c9ec0c67f7befbee5c6d283698ca1c57774f54e27f6819571df83fdc3ababce not found: ID does not exist"
Nov 04 07:39:23.789418 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.787401    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k"]
Nov 04 07:39:23.807967 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.807937    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"]
Nov 04 07:39:23.827851 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:23.827813    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt"]
Nov 04 07:39:24.096430 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:24.096396    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b8fb283-b77c-4758-9497-84ba24890da2" path="/var/lib/kubelet/pods/9b8fb283-b77c-4758-9497-84ba24890da2/volumes"
Nov 04 07:39:24.097021 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:24.096992    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8b6a0dc-8600-4ab9-923f-3d680473f39f" path="/var/lib/kubelet/pods/a8b6a0dc-8600-4ab9-923f-3d680473f39f/volumes"
Nov 04 07:39:27.638266 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:27.638238    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"
Nov 04 07:39:27.638871 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:27.638302    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"
Nov 04 07:39:33.073109 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.073065    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"]
Nov 04 07:39:33.073979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.073949    5626 cpu_manager.go:414] "containerMap: removing container" podUID="9b8fb283-b77c-4758-9497-84ba24890da2" containerName="payload-pre-processing"
Nov 04 07:39:33.073979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.073974    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b8fb283-b77c-4758-9497-84ba24890da2" containerName="payload-pre-processing"
Nov 04 07:39:33.074187 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.074002    5626 cpu_manager.go:414] "containerMap: removing container" podUID="a8b6a0dc-8600-4ab9-923f-3d680473f39f" containerName="payload-processing"
Nov 04 07:39:33.074187 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.074010    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8b6a0dc-8600-4ab9-923f-3d680473f39f" containerName="payload-processing"
Nov 04 07:39:33.074288 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.074187    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b8fb283-b77c-4758-9497-84ba24890da2" containerName="payload-pre-processing"
Nov 04 07:39:33.074288 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.074205    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8b6a0dc-8600-4ab9-923f-3d680473f39f" containerName="payload-processing"
Nov 04 07:39:33.079429 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.079410    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:33.084763 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.084736    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:39:33.087017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.086993    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:39:33.091308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.090910    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"]
Nov 04 07:39:33.092609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.092580    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:39:33.092719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.092638    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:39:33.196542 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.196507    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk68t\" (UniqueName: \"kubernetes.io/projected/4b66971a-d867-4a9f-87c7-78032e90047a-kube-api-access-gk68t\") pod \"maas-api-revoke-sub-925c0037-pvz85\" (UID: \"4b66971a-d867-4a9f-87c7-78032e90047a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:33.196720 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.196569    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-925c0037-pvz85\" (UID: \"4b66971a-d867-4a9f-87c7-78032e90047a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:33.196790 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.196772    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-925c0037-pvz85\" (UID: \"4b66971a-d867-4a9f-87c7-78032e90047a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:33.298570 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.298534    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-925c0037-pvz85\" (UID: \"4b66971a-d867-4a9f-87c7-78032e90047a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:33.298767 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.298696    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk68t\" (UniqueName: \"kubernetes.io/projected/4b66971a-d867-4a9f-87c7-78032e90047a-kube-api-access-gk68t\") pod \"maas-api-revoke-sub-925c0037-pvz85\" (UID: \"4b66971a-d867-4a9f-87c7-78032e90047a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:33.298767 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.298750    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-925c0037-pvz85\" (UID: \"4b66971a-d867-4a9f-87c7-78032e90047a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:33.299380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.299339    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-925c0037-pvz85\" (UID: \"4b66971a-d867-4a9f-87c7-78032e90047a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:33.301270 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.301246    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-925c0037-pvz85\" (UID: \"4b66971a-d867-4a9f-87c7-78032e90047a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:33.314708 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.314685    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk68t\" (UniqueName: \"kubernetes.io/projected/4b66971a-d867-4a9f-87c7-78032e90047a-kube-api-access-gk68t\") pod \"maas-api-revoke-sub-925c0037-pvz85\" (UID: \"4b66971a-d867-4a9f-87c7-78032e90047a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:33.402284 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.402188    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:33.696829 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.696798    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"]
Nov 04 07:39:33.697943 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:39:33.697918    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b66971a_d867_4a9f_87c7_78032e90047a.slice/crio-d87621c718d29c4a5964e476a64783a145382e522896855a8a7cb99fbe8c7a7b WatchSource:0}: Error finding container d87621c718d29c4a5964e476a64783a145382e522896855a8a7cb99fbe8c7a7b: Status 404 returned error can't find the container with id d87621c718d29c4a5964e476a64783a145382e522896855a8a7cb99fbe8c7a7b
Nov 04 07:39:33.883646 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:33.883613    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85" event={"ID":"4b66971a-d867-4a9f-87c7-78032e90047a","Type":"ContainerStarted","Data":"d87621c718d29c4a5964e476a64783a145382e522896855a8a7cb99fbe8c7a7b"}
Nov 04 07:39:34.892825 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:34.892712    5626 generic.go:356] "Generic (PLEG): container finished" podID="4b66971a-d867-4a9f-87c7-78032e90047a" containerID="954dabcef7f679adc2e14a26ed8731f964d9bb9e7ccb54ef3f864b4cc0be247a" exitCode=0
Nov 04 07:39:34.892825 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:34.892764    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85" event={"ID":"4b66971a-d867-4a9f-87c7-78032e90047a","Type":"ContainerDied","Data":"954dabcef7f679adc2e14a26ed8731f964d9bb9e7ccb54ef3f864b4cc0be247a"}
Nov 04 07:39:36.065828 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.065801    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:36.138588 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.138556    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-openshift-ca\") pod \"4b66971a-d867-4a9f-87c7-78032e90047a\" (UID: \"4b66971a-d867-4a9f-87c7-78032e90047a\") "
Nov 04 07:39:36.138766 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.138617    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-xks-ca\") pod \"4b66971a-d867-4a9f-87c7-78032e90047a\" (UID: \"4b66971a-d867-4a9f-87c7-78032e90047a\") "
Nov 04 07:39:36.138766 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.138718    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b66971a-d867-4a9f-87c7-78032e90047a-kube-api-access-gk68t\" (UniqueName: \"kubernetes.io/projected/4b66971a-d867-4a9f-87c7-78032e90047a-kube-api-access-gk68t\") pod \"4b66971a-d867-4a9f-87c7-78032e90047a\" (UID: \"4b66971a-d867-4a9f-87c7-78032e90047a\") "
Nov 04 07:39:36.139017 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.138976    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-openshift-ca" pod "4b66971a-d867-4a9f-87c7-78032e90047a" (UID: "4b66971a-d867-4a9f-87c7-78032e90047a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:39:36.139249 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.139226    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:36.140953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.140929    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-xks-ca" pod "4b66971a-d867-4a9f-87c7-78032e90047a" (UID: "4b66971a-d867-4a9f-87c7-78032e90047a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:39:36.140953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.140941    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b66971a-d867-4a9f-87c7-78032e90047a-kube-api-access-gk68t" pod "4b66971a-d867-4a9f-87c7-78032e90047a" (UID: "4b66971a-d867-4a9f-87c7-78032e90047a"). InnerVolumeSpecName "kube-api-access-gk68t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:36.173994 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:39:36.173957    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:39:36.241038 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.241004    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b66971a-d867-4a9f-87c7-78032e90047a-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:36.241038 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.241036    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk68t\" (UniqueName: \"kubernetes.io/projected/4b66971a-d867-4a9f-87c7-78032e90047a-kube-api-access-gk68t\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:36.909666 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.909634    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85"
Nov 04 07:39:36.909843 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.909668    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-925c0037-pvz85" event={"ID":"4b66971a-d867-4a9f-87c7-78032e90047a","Type":"ContainerDied","Data":"d87621c718d29c4a5964e476a64783a145382e522896855a8a7cb99fbe8c7a7b"}
Nov 04 07:39:36.909843 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:36.909720    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d87621c718d29c4a5964e476a64783a145382e522896855a8a7cb99fbe8c7a7b"
Nov 04 07:39:38.872609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:38.872573    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"]
Nov 04 07:39:38.873769 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:38.873711    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9" podUID="cca66d0d-442b-4138-99a3-72bc0579f077" containerName="istio-proxy" containerID="cri-o://7183bf97ffa76ae700dc3426594e2525d52f77ad1866d17819c990144998deaf" gracePeriod=30
Nov 04 07:39:38.984389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:38.972467    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"]
Nov 04 07:39:38.984389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:38.972760    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" podUID="5dcf696e-7927-43ac-a18b-f7e5128ca367" containerName="istio-proxy" containerID="cri-o://7c5b4e247628e7609d7178f73c079dc425c672fa621786055e11a85018ae540b" gracePeriod=30
Nov 04 07:39:39.005113 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:39.005058    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"]
Nov 04 07:39:39.006157 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:39.005642    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q" podUID="512d84d2-e30f-4c72-97da-cb58ccb2c8e5" containerName="istio-proxy" containerID="cri-o://42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d" gracePeriod=30
Nov 04 07:39:39.078386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:39.067973    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"]
Nov 04 07:39:39.078386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:39.068261    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl" podUID="50ee65a2-eedf-47ac-93e3-0b75218a0d7b" containerName="istio-proxy" containerID="cri-o://f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358" gracePeriod=30
Nov 04 07:39:44.103409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.102914    5626 generic.go:356] "Generic (PLEG): container finished" podID="5dcf696e-7927-43ac-a18b-f7e5128ca367" containerID="7c5b4e247628e7609d7178f73c079dc425c672fa621786055e11a85018ae540b" exitCode=0
Nov 04 07:39:44.114945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.114905    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" event={"ID":"5dcf696e-7927-43ac-a18b-f7e5128ca367","Type":"ContainerDied","Data":"7c5b4e247628e7609d7178f73c079dc425c672fa621786055e11a85018ae540b"}
Nov 04 07:39:44.116179 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.116148    5626 generic.go:356] "Generic (PLEG): container finished" podID="cca66d0d-442b-4138-99a3-72bc0579f077" containerID="7183bf97ffa76ae700dc3426594e2525d52f77ad1866d17819c990144998deaf" exitCode=0
Nov 04 07:39:44.116312 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.116194    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9" event={"ID":"cca66d0d-442b-4138-99a3-72bc0579f077","Type":"ContainerDied","Data":"7183bf97ffa76ae700dc3426594e2525d52f77ad1866d17819c990144998deaf"}
Nov 04 07:39:44.121016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.120990    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"]
Nov 04 07:39:44.247387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.244687    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:44.394387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.379296    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:39:44.421609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.414475    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cca66d0d-442b-4138-99a3-72bc0579f077-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cca66d0d-442b-4138-99a3-72bc0579f077-istiod-ca-cert\") pod \"cca66d0d-442b-4138-99a3-72bc0579f077\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") "
Nov 04 07:39:44.421609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.414586    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cca66d0d-442b-4138-99a3-72bc0579f077-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cca66d0d-442b-4138-99a3-72bc0579f077-istio-podinfo\") pod \"cca66d0d-442b-4138-99a3-72bc0579f077\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") "
Nov 04 07:39:44.421609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.414641    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-certs\") pod \"cca66d0d-442b-4138-99a3-72bc0579f077\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") "
Nov 04 07:39:44.421609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.414687    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-istio-token\" (UniqueName: \"kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-istio-token\") pod \"cca66d0d-442b-4138-99a3-72bc0579f077\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") "
Nov 04 07:39:44.421609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.414716    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-kube-api-access-2dq4z\" (UniqueName: \"kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-kube-api-access-2dq4z\") pod \"cca66d0d-442b-4138-99a3-72bc0579f077\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") "
Nov 04 07:39:44.421609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.414872    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-envoy\") pod \"cca66d0d-442b-4138-99a3-72bc0579f077\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") "
Nov 04 07:39:44.421609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.414911    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-credential-socket\") pod \"cca66d0d-442b-4138-99a3-72bc0579f077\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") "
Nov 04 07:39:44.421609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.415030    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-data\") pod \"cca66d0d-442b-4138-99a3-72bc0579f077\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") "
Nov 04 07:39:44.421609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.415073    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-socket\") pod \"cca66d0d-442b-4138-99a3-72bc0579f077\" (UID: \"cca66d0d-442b-4138-99a3-72bc0579f077\") "
Nov 04 07:39:44.422811 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.422765    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-socket" pod "cca66d0d-442b-4138-99a3-72bc0579f077" (UID: "cca66d0d-442b-4138-99a3-72bc0579f077"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.423123 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.423098    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cca66d0d-442b-4138-99a3-72bc0579f077-istiod-ca-cert" pod "cca66d0d-442b-4138-99a3-72bc0579f077" (UID: "cca66d0d-442b-4138-99a3-72bc0579f077"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:39:44.426423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.425726    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-data" pod "cca66d0d-442b-4138-99a3-72bc0579f077" (UID: "cca66d0d-442b-4138-99a3-72bc0579f077"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.426423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.425944    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-certs" pod "cca66d0d-442b-4138-99a3-72bc0579f077" (UID: "cca66d0d-442b-4138-99a3-72bc0579f077"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.426423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.426231    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-credential-socket" pod "cca66d0d-442b-4138-99a3-72bc0579f077" (UID: "cca66d0d-442b-4138-99a3-72bc0579f077"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.471385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.449896    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-istio-token" pod "cca66d0d-442b-4138-99a3-72bc0579f077" (UID: "cca66d0d-442b-4138-99a3-72bc0579f077"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:44.475954 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.475874    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cca66d0d-442b-4138-99a3-72bc0579f077-istio-podinfo" pod "cca66d0d-442b-4138-99a3-72bc0579f077" (UID: "cca66d0d-442b-4138-99a3-72bc0579f077"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:39:44.480330 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.480274    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-envoy" pod "cca66d0d-442b-4138-99a3-72bc0579f077" (UID: "cca66d0d-442b-4138-99a3-72bc0579f077"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.482594 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.482545    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-kube-api-access-2dq4z" pod "cca66d0d-442b-4138-99a3-72bc0579f077" (UID: "cca66d0d-442b-4138-99a3-72bc0579f077"). InnerVolumeSpecName "kube-api-access-2dq4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:44.523875 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.523826    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-kube-api-access-fdplg\" (UniqueName: \"kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-kube-api-access-fdplg\") pod \"5dcf696e-7927-43ac-a18b-f7e5128ca367\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") "
Nov 04 07:39:44.524033 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.523954    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-podinfo\") pod \"5dcf696e-7927-43ac-a18b-f7e5128ca367\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") "
Nov 04 07:39:44.524033 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.524024    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-token\" (UniqueName: \"kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-token\") pod \"5dcf696e-7927-43ac-a18b-f7e5128ca367\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") "
Nov 04 07:39:44.524173 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.524066    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-envoy\") pod \"5dcf696e-7927-43ac-a18b-f7e5128ca367\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") "
Nov 04 07:39:44.524173 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.524130    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-data\") pod \"5dcf696e-7927-43ac-a18b-f7e5128ca367\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") "
Nov 04 07:39:44.524289 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.524234    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-credential-socket\") pod \"5dcf696e-7927-43ac-a18b-f7e5128ca367\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") "
Nov 04 07:39:44.524289 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.524266    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-socket\") pod \"5dcf696e-7927-43ac-a18b-f7e5128ca367\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") "
Nov 04 07:39:44.524440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.524325    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-certs\") pod \"5dcf696e-7927-43ac-a18b-f7e5128ca367\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") "
Nov 04 07:39:44.524440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.524412    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5dcf696e-7927-43ac-a18b-f7e5128ca367-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dcf696e-7927-43ac-a18b-f7e5128ca367-istiod-ca-cert\") pod \"5dcf696e-7927-43ac-a18b-f7e5128ca367\" (UID: \"5dcf696e-7927-43ac-a18b-f7e5128ca367\") "
Nov 04 07:39:44.525016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.524992    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cca66d0d-442b-4138-99a3-72bc0579f077-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.525016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.525017    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.525196 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.525029    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.525196 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.525039    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dq4z\" (UniqueName: \"kubernetes.io/projected/cca66d0d-442b-4138-99a3-72bc0579f077-kube-api-access-2dq4z\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.525196 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.525049    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.525196 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.525058    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.525196 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.525067    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.525196 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.525076    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cca66d0d-442b-4138-99a3-72bc0579f077-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.525196 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.525084    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cca66d0d-442b-4138-99a3-72bc0579f077-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.525604 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.525447    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dcf696e-7927-43ac-a18b-f7e5128ca367-istiod-ca-cert" pod "5dcf696e-7927-43ac-a18b-f7e5128ca367" (UID: "5dcf696e-7927-43ac-a18b-f7e5128ca367"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:39:44.529129 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.528920    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-kube-api-access-fdplg" pod "5dcf696e-7927-43ac-a18b-f7e5128ca367" (UID: "5dcf696e-7927-43ac-a18b-f7e5128ca367"). InnerVolumeSpecName "kube-api-access-fdplg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:44.529593 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.529303    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-credential-socket" pod "5dcf696e-7927-43ac-a18b-f7e5128ca367" (UID: "5dcf696e-7927-43ac-a18b-f7e5128ca367"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.530240 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.530119    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-podinfo" pod "5dcf696e-7927-43ac-a18b-f7e5128ca367" (UID: "5dcf696e-7927-43ac-a18b-f7e5128ca367"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:39:44.530342 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.530297    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-data" pod "5dcf696e-7927-43ac-a18b-f7e5128ca367" (UID: "5dcf696e-7927-43ac-a18b-f7e5128ca367"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.530596 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.530545    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-certs" pod "5dcf696e-7927-43ac-a18b-f7e5128ca367" (UID: "5dcf696e-7927-43ac-a18b-f7e5128ca367"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.530596 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.530562    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-socket" pod "5dcf696e-7927-43ac-a18b-f7e5128ca367" (UID: "5dcf696e-7927-43ac-a18b-f7e5128ca367"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.532220 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.532074    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-envoy" pod "5dcf696e-7927-43ac-a18b-f7e5128ca367" (UID: "5dcf696e-7927-43ac-a18b-f7e5128ca367"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.532478 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.532442    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-token" pod "5dcf696e-7927-43ac-a18b-f7e5128ca367" (UID: "5dcf696e-7927-43ac-a18b-f7e5128ca367"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:44.561160 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.561130    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:44.596887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.596860    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:39:44.628304 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.628269    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.628515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.628313    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.628515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.628328    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.628515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.628340    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.628515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.628354    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5dcf696e-7927-43ac-a18b-f7e5128ca367-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.628515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.628379    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdplg\" (UniqueName: \"kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-kube-api-access-fdplg\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.628515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.628393    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.628515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.628404    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.628515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.628415    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5dcf696e-7927-43ac-a18b-f7e5128ca367-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.729227 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729127    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istiod-ca-cert\") pod \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") "
Nov 04 07:39:44.729227 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729208    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-certs\") pod \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") "
Nov 04 07:39:44.729478 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729240    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-credential-socket\") pod \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") "
Nov 04 07:39:44.729478 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729265    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-kube-api-access-mdqjk\" (UniqueName: \"kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-kube-api-access-mdqjk\") pod \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") "
Nov 04 07:39:44.729478 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729313    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-credential-socket\") pod \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") "
Nov 04 07:39:44.729478 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729338    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-podinfo\") pod \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") "
Nov 04 07:39:44.729670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729484    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-socket\") pod \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") "
Nov 04 07:39:44.729670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729514    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istiod-ca-cert\") pod \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") "
Nov 04 07:39:44.729670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729572    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-envoy\") pod \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") "
Nov 04 07:39:44.729670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729565    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-credential-socket" pod "50ee65a2-eedf-47ac-93e3-0b75218a0d7b" (UID: "50ee65a2-eedf-47ac-93e3-0b75218a0d7b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.729670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729622    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-socket\") pod \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") "
Nov 04 07:39:44.729670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729653    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istiod-ca-cert" pod "512d84d2-e30f-4c72-97da-cb58ccb2c8e5" (UID: "512d84d2-e30f-4c72-97da-cb58ccb2c8e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:39:44.729945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729672    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-token\" (UniqueName: \"kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-token\") pod \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") "
Nov 04 07:39:44.729945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729730    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-token\" (UniqueName: \"kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-token\") pod \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") "
Nov 04 07:39:44.729945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729740    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-certs" pod "512d84d2-e30f-4c72-97da-cb58ccb2c8e5" (UID: "512d84d2-e30f-4c72-97da-cb58ccb2c8e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.729945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729772    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-credential-socket" pod "512d84d2-e30f-4c72-97da-cb58ccb2c8e5" (UID: "512d84d2-e30f-4c72-97da-cb58ccb2c8e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.729945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729805    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-data\") pod \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") "
Nov 04 07:39:44.729945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729873    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-data\") pod \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") "
Nov 04 07:39:44.729945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729906    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-podinfo\") pod \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") "
Nov 04 07:39:44.729945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.729918    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-socket" pod "50ee65a2-eedf-47ac-93e3-0b75218a0d7b" (UID: "50ee65a2-eedf-47ac-93e3-0b75218a0d7b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.730344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.730003    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-envoy\") pod \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") "
Nov 04 07:39:44.730344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.730052    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-socket" pod "512d84d2-e30f-4c72-97da-cb58ccb2c8e5" (UID: "512d84d2-e30f-4c72-97da-cb58ccb2c8e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.730344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.730059    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-certs\") pod \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\" (UID: \"50ee65a2-eedf-47ac-93e3-0b75218a0d7b\") "
Nov 04 07:39:44.730344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.730163    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-data" pod "512d84d2-e30f-4c72-97da-cb58ccb2c8e5" (UID: "512d84d2-e30f-4c72-97da-cb58ccb2c8e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.730344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.730209    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-kube-api-access-w7tk8\" (UniqueName: \"kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-kube-api-access-w7tk8\") pod \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\" (UID: \"512d84d2-e30f-4c72-97da-cb58ccb2c8e5\") "
Nov 04 07:39:44.730344 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.730271    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istiod-ca-cert" pod "50ee65a2-eedf-47ac-93e3-0b75218a0d7b" (UID: "50ee65a2-eedf-47ac-93e3-0b75218a0d7b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:39:44.730703 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.730391    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-certs" pod "50ee65a2-eedf-47ac-93e3-0b75218a0d7b" (UID: "50ee65a2-eedf-47ac-93e3-0b75218a0d7b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.730953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.730923    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.731041 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.730960    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.731041 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.730978    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.731041 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.730995    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.731041 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.731010    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.731041 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.731026    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.731317 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.731041    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.731317 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.731058    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.731317 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.731073    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.732454 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.732395    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-kube-api-access-mdqjk" pod "50ee65a2-eedf-47ac-93e3-0b75218a0d7b" (UID: "50ee65a2-eedf-47ac-93e3-0b75218a0d7b"). InnerVolumeSpecName "kube-api-access-mdqjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:44.732636 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.732608    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-envoy" pod "50ee65a2-eedf-47ac-93e3-0b75218a0d7b" (UID: "50ee65a2-eedf-47ac-93e3-0b75218a0d7b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.732636 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.732624    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-data" pod "50ee65a2-eedf-47ac-93e3-0b75218a0d7b" (UID: "50ee65a2-eedf-47ac-93e3-0b75218a0d7b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.732767 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.732666    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-token" pod "50ee65a2-eedf-47ac-93e3-0b75218a0d7b" (UID: "50ee65a2-eedf-47ac-93e3-0b75218a0d7b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:44.732941 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.732918    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-podinfo" pod "50ee65a2-eedf-47ac-93e3-0b75218a0d7b" (UID: "50ee65a2-eedf-47ac-93e3-0b75218a0d7b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:39:44.732999 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.732976    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-token" pod "512d84d2-e30f-4c72-97da-cb58ccb2c8e5" (UID: "512d84d2-e30f-4c72-97da-cb58ccb2c8e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:44.733090 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.733077    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-envoy" pod "512d84d2-e30f-4c72-97da-cb58ccb2c8e5" (UID: "512d84d2-e30f-4c72-97da-cb58ccb2c8e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:39:44.733315 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.733299    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-kube-api-access-w7tk8" pod "512d84d2-e30f-4c72-97da-cb58ccb2c8e5" (UID: "512d84d2-e30f-4c72-97da-cb58ccb2c8e5"). InnerVolumeSpecName "kube-api-access-w7tk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:44.733643 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.733624    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-podinfo" pod "512d84d2-e30f-4c72-97da-cb58ccb2c8e5" (UID: "512d84d2-e30f-4c72-97da-cb58ccb2c8e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:39:44.832596 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.832562    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.832596 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.832591    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.832596 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.832602    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.832849 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.832612    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7tk8\" (UniqueName: \"kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-kube-api-access-w7tk8\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.832849 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.832624    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdqjk\" (UniqueName: \"kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-kube-api-access-mdqjk\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.832849 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.832635    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.832849 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.832645    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.832849 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.832653    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/512d84d2-e30f-4c72-97da-cb58ccb2c8e5-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:44.832849 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:44.832661    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ee65a2-eedf-47ac-93e3-0b75218a0d7b-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:45.125181 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.125148    5626 generic.go:356] "Generic (PLEG): container finished" podID="50ee65a2-eedf-47ac-93e3-0b75218a0d7b" containerID="f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358" exitCode=0
Nov 04 07:39:45.125710 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.125198    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"
Nov 04 07:39:45.125710 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.125228    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl" event={"ID":"50ee65a2-eedf-47ac-93e3-0b75218a0d7b","Type":"ContainerDied","Data":"f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358"}
Nov 04 07:39:45.125710 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.125283    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl" event={"ID":"50ee65a2-eedf-47ac-93e3-0b75218a0d7b","Type":"ContainerDied","Data":"a6c099970404c1b5aee644c217caf15b3e4e380388dc8a69e0c3f7febb3be2e7"}
Nov 04 07:39:45.125710 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.125304    5626 scope.go:122] "RemoveContainer" containerID="f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358"
Nov 04 07:39:45.126832 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.126811    5626 generic.go:356] "Generic (PLEG): container finished" podID="512d84d2-e30f-4c72-97da-cb58ccb2c8e5" containerID="42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d" exitCode=0
Nov 04 07:39:45.126914 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.126859    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q" event={"ID":"512d84d2-e30f-4c72-97da-cb58ccb2c8e5","Type":"ContainerDied","Data":"42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d"}
Nov 04 07:39:45.126963 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.126871    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"
Nov 04 07:39:45.127044 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.126916    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q" event={"ID":"512d84d2-e30f-4c72-97da-cb58ccb2c8e5","Type":"ContainerDied","Data":"4c51d2aa37cd53ec0f487519c608f5e5fda818afa353b9e2e79065c87ce9158c"}
Nov 04 07:39:45.130731 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.130707    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"
Nov 04 07:39:45.130823 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.130799    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9" event={"ID":"cca66d0d-442b-4138-99a3-72bc0579f077","Type":"ContainerDied","Data":"d3646f5f7ebfb78c83dae827cf72e341e8b8a8faf32231544fe455d59f6d06a9"}
Nov 04 07:39:45.132255 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.132232    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq" event={"ID":"5dcf696e-7927-43ac-a18b-f7e5128ca367","Type":"ContainerDied","Data":"8f4a22f18554c66b6575d76fce38a3c944a647d95189a71fcb560cec0f4b01c2"}
Nov 04 07:39:45.132351 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.132257    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"
Nov 04 07:39:45.136668 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.136650    5626 scope.go:122] "RemoveContainer" containerID="f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358"
Nov 04 07:39:45.136928 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:39:45.136907    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358\": container with ID starting with f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358 not found: ID does not exist" containerID="f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358"
Nov 04 07:39:45.137016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.136935    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358"} err="failed to get container status \"f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358\": rpc error: code = NotFound desc = could not find container \"f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358\": container with ID starting with f80ca3a2a1c465cdc085baacdee5c7335c0d9df06d726a4449f4af448f48c358 not found: ID does not exist"
Nov 04 07:39:45.137016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.136952    5626 scope.go:122] "RemoveContainer" containerID="42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d"
Nov 04 07:39:45.147981 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.147963    5626 scope.go:122] "RemoveContainer" containerID="42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d"
Nov 04 07:39:45.148273 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:39:45.148254    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d\": container with ID starting with 42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d not found: ID does not exist" containerID="42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d"
Nov 04 07:39:45.148314 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.148280    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d"} err="failed to get container status \"42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d\": rpc error: code = NotFound desc = could not find container \"42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d\": container with ID starting with 42f52c0e7d80c16bccd134fad26c64b270f4d65a21dbe09eae45977c2753d62d not found: ID does not exist"
Nov 04 07:39:45.148314 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.148297    5626 scope.go:122] "RemoveContainer" containerID="7183bf97ffa76ae700dc3426594e2525d52f77ad1866d17819c990144998deaf"
Nov 04 07:39:45.158911 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.158889    5626 scope.go:122] "RemoveContainer" containerID="7c5b4e247628e7609d7178f73c079dc425c672fa621786055e11a85018ae540b"
Nov 04 07:39:45.160279 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.160253    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"]
Nov 04 07:39:45.164301 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.164279    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q"]
Nov 04 07:39:45.176684 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.176632    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"]
Nov 04 07:39:45.181455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.181435    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl"]
Nov 04 07:39:45.193069 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.193049    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"]
Nov 04 07:39:45.199290 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.199271    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9"]
Nov 04 07:39:45.211614 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.211594    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"]
Nov 04 07:39:45.216586 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:45.216565    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq"]
Nov 04 07:39:46.094888 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:46.094853    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50ee65a2-eedf-47ac-93e3-0b75218a0d7b" path="/var/lib/kubelet/pods/50ee65a2-eedf-47ac-93e3-0b75218a0d7b/volumes"
Nov 04 07:39:46.095358 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:46.095343    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="512d84d2-e30f-4c72-97da-cb58ccb2c8e5" path="/var/lib/kubelet/pods/512d84d2-e30f-4c72-97da-cb58ccb2c8e5/volumes"
Nov 04 07:39:46.095796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:46.095782    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5dcf696e-7927-43ac-a18b-f7e5128ca367" path="/var/lib/kubelet/pods/5dcf696e-7927-43ac-a18b-f7e5128ca367/volumes"
Nov 04 07:39:46.096598 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:46.096582    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cca66d0d-442b-4138-99a3-72bc0579f077" path="/var/lib/kubelet/pods/cca66d0d-442b-4138-99a3-72bc0579f077/volumes"
Nov 04 07:39:49.123020 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.122971    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855" podUID="9cc5b6e5-a888-4e86-8f9d-18fc9499167c" containerName="payload-processing" containerID="cri-o://10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774" gracePeriod=25
Nov 04 07:39:49.402484 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.402455    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:39:49.495274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.495243    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-routing-overlay-volume\") pod \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") "
Nov 04 07:39:49.495527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.495346    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-provider-credentials\" (UniqueName: \"kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-provider-credentials\") pod \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") "
Nov 04 07:39:49.495527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.495450    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-plugins-config-volume\") pod \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") "
Nov 04 07:39:49.495527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.495487    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-kube-api-access-9dmzv\" (UniqueName: \"kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-kube-api-access-9dmzv\") pod \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\" (UID: \"9cc5b6e5-a888-4e86-8f9d-18fc9499167c\") "
Nov 04 07:39:49.495845 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.495813    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-routing-overlay-volume" pod "9cc5b6e5-a888-4e86-8f9d-18fc9499167c" (UID: "9cc5b6e5-a888-4e86-8f9d-18fc9499167c"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:39:49.495967 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.495947    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-plugins-config-volume" pod "9cc5b6e5-a888-4e86-8f9d-18fc9499167c" (UID: "9cc5b6e5-a888-4e86-8f9d-18fc9499167c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:39:49.496122 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.496108    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:49.496163 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.496124    5626 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-routing-overlay-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:49.497643 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.497620    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-provider-credentials" pod "9cc5b6e5-a888-4e86-8f9d-18fc9499167c" (UID: "9cc5b6e5-a888-4e86-8f9d-18fc9499167c"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:49.497751 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.497728    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-kube-api-access-9dmzv" pod "9cc5b6e5-a888-4e86-8f9d-18fc9499167c" (UID: "9cc5b6e5-a888-4e86-8f9d-18fc9499167c"). InnerVolumeSpecName "kube-api-access-9dmzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:49.597718 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.597684    5626 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-provider-credentials\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:49.597718 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:49.597712    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dmzv\" (UniqueName: \"kubernetes.io/projected/9cc5b6e5-a888-4e86-8f9d-18fc9499167c-kube-api-access-9dmzv\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:50.177794 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:50.177760    5626 generic.go:356] "Generic (PLEG): container finished" podID="9cc5b6e5-a888-4e86-8f9d-18fc9499167c" containerID="10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774" exitCode=0
Nov 04 07:39:50.178287 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:50.177815    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"
Nov 04 07:39:50.178287 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:50.177842    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855" event={"ID":"9cc5b6e5-a888-4e86-8f9d-18fc9499167c","Type":"ContainerDied","Data":"10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774"}
Nov 04 07:39:50.178287 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:50.177895    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855" event={"ID":"9cc5b6e5-a888-4e86-8f9d-18fc9499167c","Type":"ContainerDied","Data":"6af7d1d90305cd614eca38590c044c1df322b00ebc9c9d81310d5e5b13a54208"}
Nov 04 07:39:50.178287 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:50.177919    5626 scope.go:122] "RemoveContainer" containerID="10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774"
Nov 04 07:39:50.188241 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:50.188220    5626 scope.go:122] "RemoveContainer" containerID="10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774"
Nov 04 07:39:50.188574 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:39:50.188553    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774\": container with ID starting with 10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774 not found: ID does not exist" containerID="10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774"
Nov 04 07:39:50.188634 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:50.188580    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774"} err="failed to get container status \"10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774\": rpc error: code = NotFound desc = could not find container \"10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774\": container with ID starting with 10be4262d7b8fb13899cb6148ff80e2f2e70532909e8c4d769bbd61cd3ace774 not found: ID does not exist"
Nov 04 07:39:50.206634 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:50.206607    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"]
Nov 04 07:39:50.218014 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:50.217989    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-812032/payload-processing-external-model-e2e-extmod-b-812032-8489sj855"]
Nov 04 07:39:51.513051 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513016    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"]
Nov 04 07:39:51.513760 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513741    5626 cpu_manager.go:414] "containerMap: removing container" podUID="512d84d2-e30f-4c72-97da-cb58ccb2c8e5" containerName="istio-proxy"
Nov 04 07:39:51.513813 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513761    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="512d84d2-e30f-4c72-97da-cb58ccb2c8e5" containerName="istio-proxy"
Nov 04 07:39:51.513813 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513773    5626 cpu_manager.go:414] "containerMap: removing container" podUID="50ee65a2-eedf-47ac-93e3-0b75218a0d7b" containerName="istio-proxy"
Nov 04 07:39:51.513813 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513778    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50ee65a2-eedf-47ac-93e3-0b75218a0d7b" containerName="istio-proxy"
Nov 04 07:39:51.513813 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513796    5626 cpu_manager.go:414] "containerMap: removing container" podUID="cca66d0d-442b-4138-99a3-72bc0579f077" containerName="istio-proxy"
Nov 04 07:39:51.513813 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513802    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cca66d0d-442b-4138-99a3-72bc0579f077" containerName="istio-proxy"
Nov 04 07:39:51.513813 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513811    5626 cpu_manager.go:414] "containerMap: removing container" podUID="9cc5b6e5-a888-4e86-8f9d-18fc9499167c" containerName="payload-processing"
Nov 04 07:39:51.513993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513816    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9cc5b6e5-a888-4e86-8f9d-18fc9499167c" containerName="payload-processing"
Nov 04 07:39:51.513993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513839    5626 cpu_manager.go:414] "containerMap: removing container" podUID="5dcf696e-7927-43ac-a18b-f7e5128ca367" containerName="istio-proxy"
Nov 04 07:39:51.513993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513844    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5dcf696e-7927-43ac-a18b-f7e5128ca367" containerName="istio-proxy"
Nov 04 07:39:51.513993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513857    5626 cpu_manager.go:414] "containerMap: removing container" podUID="4b66971a-d867-4a9f-87c7-78032e90047a" containerName="revoke-keys"
Nov 04 07:39:51.513993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513861    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b66971a-d867-4a9f-87c7-78032e90047a" containerName="revoke-keys"
Nov 04 07:39:51.513993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513944    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="512d84d2-e30f-4c72-97da-cb58ccb2c8e5" containerName="istio-proxy"
Nov 04 07:39:51.513993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513952    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="50ee65a2-eedf-47ac-93e3-0b75218a0d7b" containerName="istio-proxy"
Nov 04 07:39:51.513993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513960    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="cca66d0d-442b-4138-99a3-72bc0579f077" containerName="istio-proxy"
Nov 04 07:39:51.513993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513967    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="9cc5b6e5-a888-4e86-8f9d-18fc9499167c" containerName="payload-processing"
Nov 04 07:39:51.513993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513978    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="5dcf696e-7927-43ac-a18b-f7e5128ca367" containerName="istio-proxy"
Nov 04 07:39:51.513993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.513984    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b66971a-d867-4a9f-87c7-78032e90047a" containerName="revoke-keys"
Nov 04 07:39:51.518953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.518931    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.525217 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.525186    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-a558a6\"/\"routing-overlay\""
Nov 04 07:39:51.525217 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.525196    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-a558a6\"/\"payload-processing-external-model-e2e--1bf677a6-dockercfg-d27r8\""
Nov 04 07:39:51.525527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.525502    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-a558a6\"/\"payload-processing-external-model-plugins-e2e-extmod-b-a558a6\""
Nov 04 07:39:51.525649 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.525598    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-a558a6\"/\"kube-root-ca.crt\""
Nov 04 07:39:51.525706 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.525660    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-a558a6\"/\"openshift-service-ca.crt\""
Nov 04 07:39:51.527158 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.527131    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-a558a6\"/\"agc-cred-45aab2112078\""
Nov 04 07:39:51.530465 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.530441    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"]
Nov 04 07:39:51.626924 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.626890    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") " pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.627070 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.627012    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4l4b\" (UniqueName: \"kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-kube-api-access-s4l4b\") pod \"payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") " pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.627165 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.627152    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") " pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.627225 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.627209    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") " pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.743384 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.735484    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4l4b\" (UniqueName: \"kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-kube-api-access-s4l4b\") pod \"payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") " pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.743384 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.735648    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") " pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.743384 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.735716    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") " pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.766433 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.761750    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") " pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.766433 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.762810    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") " pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.766433 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.763691    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") " pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.793436 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.790826    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") " pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.797384 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.794337    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4l4b\" (UniqueName: \"kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-kube-api-access-s4l4b\") pod \"payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") " pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:51.842445 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:51.842393    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:52.094663 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:52.094627    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9cc5b6e5-a888-4e86-8f9d-18fc9499167c" path="/var/lib/kubelet/pods/9cc5b6e5-a888-4e86-8f9d-18fc9499167c/volumes"
Nov 04 07:39:52.130954 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:52.130928    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"]
Nov 04 07:39:52.131252 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:39:52.131227    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod181b7abe_885a_47f4_97d3_0b40fc4bf994.slice/crio-fa7075d537df93b85cea3eca4cc2a37e0ef15c5ec6c75d726913bf4a6f519165 WatchSource:0}: Error finding container fa7075d537df93b85cea3eca4cc2a37e0ef15c5ec6c75d726913bf4a6f519165: Status 404 returned error can't find the container with id fa7075d537df93b85cea3eca4cc2a37e0ef15c5ec6c75d726913bf4a6f519165
Nov 04 07:39:52.194712 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:52.194682    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6" event={"ID":"181b7abe-885a-47f4-97d3-0b40fc4bf994","Type":"ContainerStarted","Data":"fa7075d537df93b85cea3eca4cc2a37e0ef15c5ec6c75d726913bf4a6f519165"}
Nov 04 07:39:53.204267 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:53.204228    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6" event={"ID":"181b7abe-885a-47f4-97d3-0b40fc4bf994","Type":"ContainerStarted","Data":"3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62"}
Nov 04 07:39:53.204772 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:53.204473    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:39:56.240742 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.240679    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6" podStartSLOduration=5.240662096 podStartE2EDuration="5.240662096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:39:53.236275794 +0000 UTC m=+1415.258491069" watchObservedRunningTime="2026-11-04 07:39:56.240662096 +0000 UTC m=+1418.262877387"
Nov 04 07:39:56.244195 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.244170    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"]
Nov 04 07:39:56.249208 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.249182    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:39:56.254701 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.254667    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:39:56.254836 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.254780    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:39:56.256946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.256917    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"]
Nov 04 07:39:56.312210 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.312172    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:39:56.312420 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.312240    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:39:56.334711 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.334678    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e0ee3629-fszdp\" (UID: \"75bd9dcf-c567-4dad-a161-4d045f108bc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:39:56.334877 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.334785    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkfwn\" (UniqueName: \"kubernetes.io/projected/75bd9dcf-c567-4dad-a161-4d045f108bc2-kube-api-access-qkfwn\") pod \"maas-api-revoke-sub-e0ee3629-fszdp\" (UID: \"75bd9dcf-c567-4dad-a161-4d045f108bc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:39:56.334920 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.334898    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e0ee3629-fszdp\" (UID: \"75bd9dcf-c567-4dad-a161-4d045f108bc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:39:56.436509 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.436480    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e0ee3629-fszdp\" (UID: \"75bd9dcf-c567-4dad-a161-4d045f108bc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:39:56.436704 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.436580    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e0ee3629-fszdp\" (UID: \"75bd9dcf-c567-4dad-a161-4d045f108bc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:39:56.436704 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.436658    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkfwn\" (UniqueName: \"kubernetes.io/projected/75bd9dcf-c567-4dad-a161-4d045f108bc2-kube-api-access-qkfwn\") pod \"maas-api-revoke-sub-e0ee3629-fszdp\" (UID: \"75bd9dcf-c567-4dad-a161-4d045f108bc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:39:56.437174 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.437149    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e0ee3629-fszdp\" (UID: \"75bd9dcf-c567-4dad-a161-4d045f108bc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:39:56.438989 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.438967    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e0ee3629-fszdp\" (UID: \"75bd9dcf-c567-4dad-a161-4d045f108bc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:39:56.457756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.457727    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkfwn\" (UniqueName: \"kubernetes.io/projected/75bd9dcf-c567-4dad-a161-4d045f108bc2-kube-api-access-qkfwn\") pod \"maas-api-revoke-sub-e0ee3629-fszdp\" (UID: \"75bd9dcf-c567-4dad-a161-4d045f108bc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:39:56.477803 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.477773    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"]
Nov 04 07:39:56.485767 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.485746    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:39:56.511593 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.511565    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"]
Nov 04 07:39:56.568688 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.568658    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:39:56.645663 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.639812    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-256bd16c-prww5\" (UID: \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:39:56.645663 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.640033    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-256bd16c-prww5\" (UID: \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:39:56.645663 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.640217    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prl9v\" (UniqueName: \"kubernetes.io/projected/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-kube-api-access-prl9v\") pod \"maas-api-revoke-sub-256bd16c-prww5\" (UID: \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:39:56.741310 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.741268    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prl9v\" (UniqueName: \"kubernetes.io/projected/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-kube-api-access-prl9v\") pod \"maas-api-revoke-sub-256bd16c-prww5\" (UID: \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:39:56.741557 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.741392    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-256bd16c-prww5\" (UID: \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:39:56.741557 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.741506    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-256bd16c-prww5\" (UID: \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:39:56.742027 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.742007    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-256bd16c-prww5\" (UID: \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:39:56.743875 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.743805    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-256bd16c-prww5\" (UID: \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:39:56.753788 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.753767    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prl9v\" (UniqueName: \"kubernetes.io/projected/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-kube-api-access-prl9v\") pod \"maas-api-revoke-sub-256bd16c-prww5\" (UID: \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:39:56.800576 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.800547    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:39:56.851038 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:56.851005    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"]
Nov 04 07:39:56.852518 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:39:56.852480    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75bd9dcf_c567_4dad_a161_4d045f108bc2.slice/crio-ff1cfa254c1f1285d0a075211b95cfb6d15a2112ae1592455a4357f88c8ca870 WatchSource:0}: Error finding container ff1cfa254c1f1285d0a075211b95cfb6d15a2112ae1592455a4357f88c8ca870: Status 404 returned error can't find the container with id ff1cfa254c1f1285d0a075211b95cfb6d15a2112ae1592455a4357f88c8ca870
Nov 04 07:39:57.083298 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:57.083271    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"]
Nov 04 07:39:57.084650 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:39:57.084624    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ca75d4f_c0f9_4e86_bb1d_d455f909acb8.slice/crio-27c56dfd61fec07a9d80e83fef1cfce4adb6e8cb6cb6b3c26105055166d734e6 WatchSource:0}: Error finding container 27c56dfd61fec07a9d80e83fef1cfce4adb6e8cb6cb6b3c26105055166d734e6: Status 404 returned error can't find the container with id 27c56dfd61fec07a9d80e83fef1cfce4adb6e8cb6cb6b3c26105055166d734e6
Nov 04 07:39:57.239210 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:57.239174    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5" event={"ID":"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8","Type":"ContainerStarted","Data":"27c56dfd61fec07a9d80e83fef1cfce4adb6e8cb6cb6b3c26105055166d734e6"}
Nov 04 07:39:57.241428 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:57.241394    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp" event={"ID":"75bd9dcf-c567-4dad-a161-4d045f108bc2","Type":"ContainerStarted","Data":"da8c0448965d7cd68275ee3f12f4f5e6684d1fa0dc734e1c89b2fa62381869f6"}
Nov 04 07:39:57.241863 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:57.241451    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp" event={"ID":"75bd9dcf-c567-4dad-a161-4d045f108bc2","Type":"ContainerStarted","Data":"ff1cfa254c1f1285d0a075211b95cfb6d15a2112ae1592455a4357f88c8ca870"}
Nov 04 07:39:57.267153 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:57.267099    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp" podStartSLOduration=1.267085092 podStartE2EDuration="1.267085092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:39:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:39:57.265192299 +0000 UTC m=+1419.287407559" watchObservedRunningTime="2026-11-04 07:39:57.267085092 +0000 UTC m=+1419.289300352"
Nov 04 07:39:58.251196 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:58.251158    5626 generic.go:356] "Generic (PLEG): container finished" podID="5ca75d4f-c0f9-4e86-bb1d-d455f909acb8" containerID="4cc928290994b75b17b510a2cb8ae3db3b1bc6eb443602c4837e985947fb2c29" exitCode=0
Nov 04 07:39:58.251704 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:58.251253    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5" event={"ID":"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8","Type":"ContainerDied","Data":"4cc928290994b75b17b510a2cb8ae3db3b1bc6eb443602c4837e985947fb2c29"}
Nov 04 07:39:58.253066 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:58.253039    5626 generic.go:356] "Generic (PLEG): container finished" podID="75bd9dcf-c567-4dad-a161-4d045f108bc2" containerID="da8c0448965d7cd68275ee3f12f4f5e6684d1fa0dc734e1c89b2fa62381869f6" exitCode=0
Nov 04 07:39:58.253196 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:58.253078    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp" event={"ID":"75bd9dcf-c567-4dad-a161-4d045f108bc2","Type":"ContainerDied","Data":"da8c0448965d7cd68275ee3f12f4f5e6684d1fa0dc734e1c89b2fa62381869f6"}
Nov 04 07:39:59.475473 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.475445    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:39:59.479252 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.479224    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:39:59.577553 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.577518    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75bd9dcf-c567-4dad-a161-4d045f108bc2-kube-api-access-qkfwn\" (UniqueName: \"kubernetes.io/projected/75bd9dcf-c567-4dad-a161-4d045f108bc2-kube-api-access-qkfwn\") pod \"75bd9dcf-c567-4dad-a161-4d045f108bc2\" (UID: \"75bd9dcf-c567-4dad-a161-4d045f108bc2\") "
Nov 04 07:39:59.577761 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.577576    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-kube-api-access-prl9v\" (UniqueName: \"kubernetes.io/projected/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-kube-api-access-prl9v\") pod \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\" (UID: \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\") "
Nov 04 07:39:59.577761 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.577675    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-xks-ca\") pod \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\" (UID: \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\") "
Nov 04 07:39:59.577891 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.577875    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-xks-ca\") pod \"75bd9dcf-c567-4dad-a161-4d045f108bc2\" (UID: \"75bd9dcf-c567-4dad-a161-4d045f108bc2\") "
Nov 04 07:39:59.577951 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.577928    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-openshift-ca\") pod \"75bd9dcf-c567-4dad-a161-4d045f108bc2\" (UID: \"75bd9dcf-c567-4dad-a161-4d045f108bc2\") "
Nov 04 07:39:59.578007 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.577962    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-openshift-ca\") pod \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\" (UID: \"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8\") "
Nov 04 07:39:59.578477 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.578440    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-openshift-ca" pod "5ca75d4f-c0f9-4e86-bb1d-d455f909acb8" (UID: "5ca75d4f-c0f9-4e86-bb1d-d455f909acb8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:39:59.578688 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.578660    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:59.578836 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.578815    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-openshift-ca" pod "75bd9dcf-c567-4dad-a161-4d045f108bc2" (UID: "75bd9dcf-c567-4dad-a161-4d045f108bc2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:39:59.580240 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.580214    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-kube-api-access-prl9v" pod "5ca75d4f-c0f9-4e86-bb1d-d455f909acb8" (UID: "5ca75d4f-c0f9-4e86-bb1d-d455f909acb8"). InnerVolumeSpecName "kube-api-access-prl9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:59.580386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.580281    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75bd9dcf-c567-4dad-a161-4d045f108bc2-kube-api-access-qkfwn" pod "75bd9dcf-c567-4dad-a161-4d045f108bc2" (UID: "75bd9dcf-c567-4dad-a161-4d045f108bc2"). InnerVolumeSpecName "kube-api-access-qkfwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:39:59.580560 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.580519    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-xks-ca" pod "75bd9dcf-c567-4dad-a161-4d045f108bc2" (UID: "75bd9dcf-c567-4dad-a161-4d045f108bc2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:39:59.580560 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.580527    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-xks-ca" pod "5ca75d4f-c0f9-4e86-bb1d-d455f909acb8" (UID: "5ca75d4f-c0f9-4e86-bb1d-d455f909acb8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:39:59.680574 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.680540    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:59.680574 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.680567    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:59.680574 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.680577    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/75bd9dcf-c567-4dad-a161-4d045f108bc2-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:59.680800 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.680588    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkfwn\" (UniqueName: \"kubernetes.io/projected/75bd9dcf-c567-4dad-a161-4d045f108bc2-kube-api-access-qkfwn\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:39:59.680800 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:39:59.680597    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prl9v\" (UniqueName: \"kubernetes.io/projected/5ca75d4f-c0f9-4e86-bb1d-d455f909acb8-kube-api-access-prl9v\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:00.272593 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:00.272565    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5"
Nov 04 07:40:00.272774 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:00.272606    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-256bd16c-prww5" event={"ID":"5ca75d4f-c0f9-4e86-bb1d-d455f909acb8","Type":"ContainerDied","Data":"27c56dfd61fec07a9d80e83fef1cfce4adb6e8cb6cb6b3c26105055166d734e6"}
Nov 04 07:40:00.272774 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:00.272669    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27c56dfd61fec07a9d80e83fef1cfce4adb6e8cb6cb6b3c26105055166d734e6"
Nov 04 07:40:00.274171 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:00.274149    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp" event={"ID":"75bd9dcf-c567-4dad-a161-4d045f108bc2","Type":"ContainerDied","Data":"ff1cfa254c1f1285d0a075211b95cfb6d15a2112ae1592455a4357f88c8ca870"}
Nov 04 07:40:00.274171 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:00.274168    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e0ee3629-fszdp"
Nov 04 07:40:00.274171 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:00.274183    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff1cfa254c1f1285d0a075211b95cfb6d15a2112ae1592455a4357f88c8ca870"
Nov 04 07:40:04.214820 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:04.214791    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:40:06.173186 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:40:06.173146    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:40:08.445082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:08.445032    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"]
Nov 04 07:40:08.981293 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:08.981260    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"]
Nov 04 07:40:13.446031 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.445984    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x" podUID="316c48f5-de99-4977-a9a0-1ae5070ba20d" containerName="payload-processing" containerID="cri-o://52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183" gracePeriod=25
Nov 04 07:40:13.738608 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.738580    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:40:13.788215 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.788152    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-plugins-config-volume\") pod \"316c48f5-de99-4977-a9a0-1ae5070ba20d\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") "
Nov 04 07:40:13.788453 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.788295    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-provider-credentials\" (UniqueName: \"kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-provider-credentials\") pod \"316c48f5-de99-4977-a9a0-1ae5070ba20d\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") "
Nov 04 07:40:13.788453 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.788382    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-routing-overlay-volume\") pod \"316c48f5-de99-4977-a9a0-1ae5070ba20d\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") "
Nov 04 07:40:13.788596 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.788480    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-kube-api-access-pkmvc\" (UniqueName: \"kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-kube-api-access-pkmvc\") pod \"316c48f5-de99-4977-a9a0-1ae5070ba20d\" (UID: \"316c48f5-de99-4977-a9a0-1ae5070ba20d\") "
Nov 04 07:40:13.788727 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.788675    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-plugins-config-volume" pod "316c48f5-de99-4977-a9a0-1ae5070ba20d" (UID: "316c48f5-de99-4977-a9a0-1ae5070ba20d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:40:13.789098 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.789061    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-routing-overlay-volume" pod "316c48f5-de99-4977-a9a0-1ae5070ba20d" (UID: "316c48f5-de99-4977-a9a0-1ae5070ba20d"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:40:13.789098 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.789081    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:13.791074 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.791049    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-provider-credentials" pod "316c48f5-de99-4977-a9a0-1ae5070ba20d" (UID: "316c48f5-de99-4977-a9a0-1ae5070ba20d"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:40:13.791519 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.791499    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-kube-api-access-pkmvc" pod "316c48f5-de99-4977-a9a0-1ae5070ba20d" (UID: "316c48f5-de99-4977-a9a0-1ae5070ba20d"). InnerVolumeSpecName "kube-api-access-pkmvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:40:13.890649 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.890617    5626 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-provider-credentials\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:13.890649 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.890644    5626 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/316c48f5-de99-4977-a9a0-1ae5070ba20d-routing-overlay-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:13.890649 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.890654    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkmvc\" (UniqueName: \"kubernetes.io/projected/316c48f5-de99-4977-a9a0-1ae5070ba20d-kube-api-access-pkmvc\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:13.981726 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:13.981627    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w" podUID="a4c782a9-861e-4733-9c6e-41a55b0610b4" containerName="payload-processing" containerID="cri-o://d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848" gracePeriod=25
Nov 04 07:40:14.242535 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.242507    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"
Nov 04 07:40:14.295046 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.295013    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4c782a9-861e-4733-9c6e-41a55b0610b4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4c782a9-861e-4733-9c6e-41a55b0610b4-plugins-config-volume\") pod \"a4c782a9-861e-4733-9c6e-41a55b0610b4\" (UID: \"a4c782a9-861e-4733-9c6e-41a55b0610b4\") "
Nov 04 07:40:14.295248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.295237    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4c782a9-861e-4733-9c6e-41a55b0610b4-kube-api-access-p9s9g\" (UniqueName: \"kubernetes.io/projected/a4c782a9-861e-4733-9c6e-41a55b0610b4-kube-api-access-p9s9g\") pod \"a4c782a9-861e-4733-9c6e-41a55b0610b4\" (UID: \"a4c782a9-861e-4733-9c6e-41a55b0610b4\") "
Nov 04 07:40:14.295488 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.295462    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4c782a9-861e-4733-9c6e-41a55b0610b4-plugins-config-volume" pod "a4c782a9-861e-4733-9c6e-41a55b0610b4" (UID: "a4c782a9-861e-4733-9c6e-41a55b0610b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:40:14.295687 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.295669    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4c782a9-861e-4733-9c6e-41a55b0610b4-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:14.297448 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.297423    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4c782a9-861e-4733-9c6e-41a55b0610b4-kube-api-access-p9s9g" pod "a4c782a9-861e-4733-9c6e-41a55b0610b4" (UID: "a4c782a9-861e-4733-9c6e-41a55b0610b4"). InnerVolumeSpecName "kube-api-access-p9s9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:40:14.397243 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.397212    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9s9g\" (UniqueName: \"kubernetes.io/projected/a4c782a9-861e-4733-9c6e-41a55b0610b4-kube-api-access-p9s9g\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:14.424858 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.424827    5626 generic.go:356] "Generic (PLEG): container finished" podID="a4c782a9-861e-4733-9c6e-41a55b0610b4" containerID="d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848" exitCode=0
Nov 04 07:40:14.425045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.424871    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"
Nov 04 07:40:14.425045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.424910    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w" event={"ID":"a4c782a9-861e-4733-9c6e-41a55b0610b4","Type":"ContainerDied","Data":"d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848"}
Nov 04 07:40:14.425045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.424966    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w" event={"ID":"a4c782a9-861e-4733-9c6e-41a55b0610b4","Type":"ContainerDied","Data":"4e8f00d033991f934f81c091e4fbe035eb0bdf3dcb210a4fe12491706e2e81f3"}
Nov 04 07:40:14.425045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.424994    5626 scope.go:122] "RemoveContainer" containerID="d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848"
Nov 04 07:40:14.426577 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.426551    5626 generic.go:356] "Generic (PLEG): container finished" podID="316c48f5-de99-4977-a9a0-1ae5070ba20d" containerID="52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183" exitCode=0
Nov 04 07:40:14.426714 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.426592    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x" event={"ID":"316c48f5-de99-4977-a9a0-1ae5070ba20d","Type":"ContainerDied","Data":"52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183"}
Nov 04 07:40:14.426714 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.426621    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"
Nov 04 07:40:14.426714 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.426635    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x" event={"ID":"316c48f5-de99-4977-a9a0-1ae5070ba20d","Type":"ContainerDied","Data":"70cb0f1cdb873dad4807749de6b935cd4494279fafe1155e65ef6fa016fdd8f1"}
Nov 04 07:40:14.437066 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.437042    5626 scope.go:122] "RemoveContainer" containerID="d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848"
Nov 04 07:40:14.437320 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:40:14.437302    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848\": container with ID starting with d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848 not found: ID does not exist" containerID="d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848"
Nov 04 07:40:14.437392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.437329    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848"} err="failed to get container status \"d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848\": rpc error: code = NotFound desc = could not find container \"d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848\": container with ID starting with d7b60e1d465669b8f819d91800ee096326271ad8082d1ae696a951fd5420b848 not found: ID does not exist"
Nov 04 07:40:14.437392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.437346    5626 scope.go:122] "RemoveContainer" containerID="52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183"
Nov 04 07:40:14.446695 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.446675    5626 scope.go:122] "RemoveContainer" containerID="52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183"
Nov 04 07:40:14.447018 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:40:14.446965    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183\": container with ID starting with 52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183 not found: ID does not exist" containerID="52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183"
Nov 04 07:40:14.447018 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.446989    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183"} err="failed to get container status \"52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183\": rpc error: code = NotFound desc = could not find container \"52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183\": container with ID starting with 52a535b97b1cb0237dd946d2e4a029bf31b0222f4f4a6da3373ee6c5a295e183 not found: ID does not exist"
Nov 04 07:40:14.454226 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.454204    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"]
Nov 04 07:40:14.457505 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.457485    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-be2a74/payload-processing-external-model-e2e-extmod-b-be2a74-76f7svl9x"]
Nov 04 07:40:14.470242 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.470220    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"]
Nov 04 07:40:14.473032 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:14.473012    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w"]
Nov 04 07:40:16.094493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:16.094458    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="316c48f5-de99-4977-a9a0-1ae5070ba20d" path="/var/lib/kubelet/pods/316c48f5-de99-4977-a9a0-1ae5070ba20d/volumes"
Nov 04 07:40:16.094939 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:16.094923    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4c782a9-861e-4733-9c6e-41a55b0610b4" path="/var/lib/kubelet/pods/a4c782a9-861e-4733-9c6e-41a55b0610b4/volumes"
Nov 04 07:40:24.094819 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.094787    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"]
Nov 04 07:40:24.095416 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.095400    5626 cpu_manager.go:414] "containerMap: removing container" podUID="75bd9dcf-c567-4dad-a161-4d045f108bc2" containerName="revoke-keys"
Nov 04 07:40:24.095416 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.095416    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75bd9dcf-c567-4dad-a161-4d045f108bc2" containerName="revoke-keys"
Nov 04 07:40:24.095530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.095452    5626 cpu_manager.go:414] "containerMap: removing container" podUID="5ca75d4f-c0f9-4e86-bb1d-d455f909acb8" containerName="revoke-keys"
Nov 04 07:40:24.095530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.095458    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ca75d4f-c0f9-4e86-bb1d-d455f909acb8" containerName="revoke-keys"
Nov 04 07:40:24.095530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.095467    5626 cpu_manager.go:414] "containerMap: removing container" podUID="316c48f5-de99-4977-a9a0-1ae5070ba20d" containerName="payload-processing"
Nov 04 07:40:24.095530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.095473    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="316c48f5-de99-4977-a9a0-1ae5070ba20d" containerName="payload-processing"
Nov 04 07:40:24.095530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.095491    5626 cpu_manager.go:414] "containerMap: removing container" podUID="a4c782a9-861e-4733-9c6e-41a55b0610b4" containerName="payload-processing"
Nov 04 07:40:24.095530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.095496    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4c782a9-861e-4733-9c6e-41a55b0610b4" containerName="payload-processing"
Nov 04 07:40:24.095743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.095583    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ca75d4f-c0f9-4e86-bb1d-d455f909acb8" containerName="revoke-keys"
Nov 04 07:40:24.095743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.095594    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="316c48f5-de99-4977-a9a0-1ae5070ba20d" containerName="payload-processing"
Nov 04 07:40:24.095743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.095603    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4c782a9-861e-4733-9c6e-41a55b0610b4" containerName="payload-processing"
Nov 04 07:40:24.095743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.095611    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="75bd9dcf-c567-4dad-a161-4d045f108bc2" containerName="revoke-keys"
Nov 04 07:40:24.101270 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.101244    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.105274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.105248    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"]
Nov 04 07:40:24.108232 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.108207    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Nov 04 07:40:24.109813 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.109758    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Nov 04 07:40:24.110036 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.110020    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Nov 04 07:40:24.115123 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.114436    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Nov 04 07:40:24.192423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.180528    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-96131c7d0580\""
Nov 04 07:40:24.192423 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.180630    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-c74sz\""
Nov 04 07:40:24.237756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.235966    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-provider-credentials\") pod \"payload-processing-external-model-8474b49dbc-pxhg8\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.237756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.236027    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-routing-overlay-volume\") pod \"payload-processing-external-model-8474b49dbc-pxhg8\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.237756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.236225    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9bwz\" (UniqueName: \"kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-kube-api-access-b9bwz\") pod \"payload-processing-external-model-8474b49dbc-pxhg8\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.237756 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.236376    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-plugins-config-volume\") pod \"payload-processing-external-model-8474b49dbc-pxhg8\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.337404 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.337321    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-plugins-config-volume\") pod \"payload-processing-external-model-8474b49dbc-pxhg8\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.337591 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.337425    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-provider-credentials\") pod \"payload-processing-external-model-8474b49dbc-pxhg8\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.337591 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.337467    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-routing-overlay-volume\") pod \"payload-processing-external-model-8474b49dbc-pxhg8\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.337741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.337614    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9bwz\" (UniqueName: \"kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-kube-api-access-b9bwz\") pod \"payload-processing-external-model-8474b49dbc-pxhg8\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.338651 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.338623    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-routing-overlay-volume\") pod \"payload-processing-external-model-8474b49dbc-pxhg8\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.339101 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.339075    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-plugins-config-volume\") pod \"payload-processing-external-model-8474b49dbc-pxhg8\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.342591 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.342566    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-provider-credentials\") pod \"payload-processing-external-model-8474b49dbc-pxhg8\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.355552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.355485    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9bwz\" (UniqueName: \"kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-kube-api-access-b9bwz\") pod \"payload-processing-external-model-8474b49dbc-pxhg8\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.424397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.424325    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:24.712324 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.712294    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"]
Nov 04 07:40:24.712843 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:40:24.712821    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6004603c_c43c_41ea_9434_82954cb86dbe.slice/crio-b72dd530638f4c253f6c3b2aaec44f0ce7d266e8fab387002a3ee22b3bae1bac WatchSource:0}: Error finding container b72dd530638f4c253f6c3b2aaec44f0ce7d266e8fab387002a3ee22b3bae1bac: Status 404 returned error can't find the container with id b72dd530638f4c253f6c3b2aaec44f0ce7d266e8fab387002a3ee22b3bae1bac
Nov 04 07:40:24.900419 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.900378    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"]
Nov 04 07:40:24.904915 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.904889    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:24.914315 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.914284    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-15dc10864e12\""
Nov 04 07:40:24.915645 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.915617    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"]
Nov 04 07:40:24.949317 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.949278    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-provider-credentials\") pod \"payload-processing-external-model-7476dd8d5d-n9wd4\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") " pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:24.949557 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.949408    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-routing-overlay-volume\") pod \"payload-processing-external-model-7476dd8d5d-n9wd4\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") " pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:24.949557 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.949459    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-plugins-config-volume\") pod \"payload-processing-external-model-7476dd8d5d-n9wd4\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") " pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:24.949696 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:24.949581    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrtx6\" (UniqueName: \"kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-kube-api-access-nrtx6\") pod \"payload-processing-external-model-7476dd8d5d-n9wd4\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") " pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:25.050316 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.050275    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrtx6\" (UniqueName: \"kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-kube-api-access-nrtx6\") pod \"payload-processing-external-model-7476dd8d5d-n9wd4\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") " pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:25.050525 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.050432    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-provider-credentials\") pod \"payload-processing-external-model-7476dd8d5d-n9wd4\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") " pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:25.050525 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.050502    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-routing-overlay-volume\") pod \"payload-processing-external-model-7476dd8d5d-n9wd4\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") " pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:25.050609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.050543    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-plugins-config-volume\") pod \"payload-processing-external-model-7476dd8d5d-n9wd4\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") " pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:25.051190 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.051165    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-routing-overlay-volume\") pod \"payload-processing-external-model-7476dd8d5d-n9wd4\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") " pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:25.051354 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.051331    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-plugins-config-volume\") pod \"payload-processing-external-model-7476dd8d5d-n9wd4\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") " pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:25.052958 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.052941    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-provider-credentials\") pod \"payload-processing-external-model-7476dd8d5d-n9wd4\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") " pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:25.067722 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.067688    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrtx6\" (UniqueName: \"kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-kube-api-access-nrtx6\") pod \"payload-processing-external-model-7476dd8d5d-n9wd4\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") " pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:25.222067 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.221759    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:25.516315 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.516278    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"]
Nov 04 07:40:25.517722 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:40:25.517698    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f930022_3f89_466e_bbd9_f545fdc8353c.slice/crio-71e6194609ffee45fa33d99668e60454facbbbc57b72b4d34a2bb9be7a4ae273 WatchSource:0}: Error finding container 71e6194609ffee45fa33d99668e60454facbbbc57b72b4d34a2bb9be7a4ae273: Status 404 returned error can't find the container with id 71e6194609ffee45fa33d99668e60454facbbbc57b72b4d34a2bb9be7a4ae273
Nov 04 07:40:25.545565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.545528    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4" event={"ID":"0f930022-3f89-466e-bbd9-f545fdc8353c","Type":"ContainerStarted","Data":"71e6194609ffee45fa33d99668e60454facbbbc57b72b4d34a2bb9be7a4ae273"}
Nov 04 07:40:25.547255 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.547199    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8" event={"ID":"6004603c-c43c-41ea-9434-82954cb86dbe","Type":"ContainerStarted","Data":"5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905"}
Nov 04 07:40:25.547386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.547267    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8" event={"ID":"6004603c-c43c-41ea-9434-82954cb86dbe","Type":"ContainerStarted","Data":"b72dd530638f4c253f6c3b2aaec44f0ce7d266e8fab387002a3ee22b3bae1bac"}
Nov 04 07:40:25.547386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.547301    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:25.572306 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:25.572265    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8" podStartSLOduration=1.572247726 podStartE2EDuration="1.572247726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:40:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:40:25.569066952 +0000 UTC m=+1447.591282213" watchObservedRunningTime="2026-11-04 07:40:25.572247726 +0000 UTC m=+1447.594462986"
Nov 04 07:40:26.556749 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:26.556707    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4" event={"ID":"0f930022-3f89-466e-bbd9-f545fdc8353c","Type":"ContainerStarted","Data":"1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376"}
Nov 04 07:40:26.557193 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:26.556908    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:26.581452 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:26.581395    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4" podStartSLOduration=2.581374828 podStartE2EDuration="2.581374828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:40:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:40:26.57809156 +0000 UTC m=+1448.600306821" watchObservedRunningTime="2026-11-04 07:40:26.581374828 +0000 UTC m=+1448.603590079"
Nov 04 07:40:31.671554 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:31.671468    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"]
Nov 04 07:40:36.255557 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:40:36.255482    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:40:36.560578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:36.560540    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:36.671808 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:36.671770    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp" podUID="daaf0ef4-096a-4b48-9903-dabc9ec22ab9" containerName="payload-pre-processing" containerID="cri-o://f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c" gracePeriod=25
Nov 04 07:40:37.157866 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:37.157769    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"
Nov 04 07:40:37.176066 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:37.175996    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-kube-api-access-qzggd\" (UniqueName: \"kubernetes.io/projected/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-kube-api-access-qzggd\") pod \"daaf0ef4-096a-4b48-9903-dabc9ec22ab9\" (UID: \"daaf0ef4-096a-4b48-9903-dabc9ec22ab9\") "
Nov 04 07:40:37.176271 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:37.176212    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-plugins-config-volume\") pod \"daaf0ef4-096a-4b48-9903-dabc9ec22ab9\" (UID: \"daaf0ef4-096a-4b48-9903-dabc9ec22ab9\") "
Nov 04 07:40:37.176869 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:37.176823    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-plugins-config-volume" pod "daaf0ef4-096a-4b48-9903-dabc9ec22ab9" (UID: "daaf0ef4-096a-4b48-9903-dabc9ec22ab9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:40:37.181922 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:37.181888    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-kube-api-access-qzggd" pod "daaf0ef4-096a-4b48-9903-dabc9ec22ab9" (UID: "daaf0ef4-096a-4b48-9903-dabc9ec22ab9"). InnerVolumeSpecName "kube-api-access-qzggd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:40:37.278160 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:37.277939    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:37.278160 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:37.277991    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzggd\" (UniqueName: \"kubernetes.io/projected/daaf0ef4-096a-4b48-9903-dabc9ec22ab9-kube-api-access-qzggd\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:37.575387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:37.574558    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:40:37.667392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:37.663705    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"]
Nov 04 07:40:37.920445 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:37.914331    5626 generic.go:356] "Generic (PLEG): container finished" podID="daaf0ef4-096a-4b48-9903-dabc9ec22ab9" containerID="f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c" exitCode=0
Nov 04 07:40:37.950530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:37.914651    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"
Nov 04 07:40:38.027435 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:38.027380    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp" event={"ID":"daaf0ef4-096a-4b48-9903-dabc9ec22ab9","Type":"ContainerDied","Data":"f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c"}
Nov 04 07:40:38.027648 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:38.027479    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp" event={"ID":"daaf0ef4-096a-4b48-9903-dabc9ec22ab9","Type":"ContainerDied","Data":"b498a3e04a4bd2ea173aea596a4ff5f57b8b24e42f1c5049fbfb7857949540c9"}
Nov 04 07:40:38.027648 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:38.027514    5626 scope.go:122] "RemoveContainer" containerID="f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c"
Nov 04 07:40:38.293669 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:38.266520    5626 scope.go:122] "RemoveContainer" containerID="f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c"
Nov 04 07:40:38.293669 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:38.266799    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"]
Nov 04 07:40:38.293669 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:40:38.276539    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c\": container with ID starting with f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c not found: ID does not exist" containerID="f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c"
Nov 04 07:40:38.293669 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:38.276586    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c"} err="failed to get container status \"f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c\": rpc error: code = NotFound desc = could not find container \"f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c\": container with ID starting with f6032e5aa10c1dd75c4f318b99437fc9cb3cc0bfe3158ab97572284431273b7c not found: ID does not exist"
Nov 04 07:40:38.387389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:38.344277    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp"]
Nov 04 07:40:38.845389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:38.844536    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:40:40.095421 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.093949    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="daaf0ef4-096a-4b48-9903-dabc9ec22ab9" path="/var/lib/kubelet/pods/daaf0ef4-096a-4b48-9903-dabc9ec22ab9/volumes"
Nov 04 07:40:40.821926 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.821886    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"]
Nov 04 07:40:40.822620 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.822603    5626 cpu_manager.go:414] "containerMap: removing container" podUID="daaf0ef4-096a-4b48-9903-dabc9ec22ab9" containerName="payload-pre-processing"
Nov 04 07:40:40.822675 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.822621    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daaf0ef4-096a-4b48-9903-dabc9ec22ab9" containerName="payload-pre-processing"
Nov 04 07:40:40.822823 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.822809    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="daaf0ef4-096a-4b48-9903-dabc9ec22ab9" containerName="payload-pre-processing"
Nov 04 07:40:40.827629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.827598    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:40.832403 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.832175    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:40:40.833433 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.833409    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:40:40.833551 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.833503    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:40:40.838834 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.838808    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"]
Nov 04 07:40:40.840730 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.839144    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:40:40.945690 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.945649    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-03bf3578-g4zm2\" (UID: \"a8b5311c-e2ce-4717-9009-8e53b9488740\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:40.945904 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.945712    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-xks-ca\") pod \"maas-api-revoke-sub-03bf3578-g4zm2\" (UID: \"a8b5311c-e2ce-4717-9009-8e53b9488740\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:40.945977 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:40.945920    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcwln\" (UniqueName: \"kubernetes.io/projected/a8b5311c-e2ce-4717-9009-8e53b9488740-kube-api-access-lcwln\") pod \"maas-api-revoke-sub-03bf3578-g4zm2\" (UID: \"a8b5311c-e2ce-4717-9009-8e53b9488740\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:41.047615 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:41.047571    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcwln\" (UniqueName: \"kubernetes.io/projected/a8b5311c-e2ce-4717-9009-8e53b9488740-kube-api-access-lcwln\") pod \"maas-api-revoke-sub-03bf3578-g4zm2\" (UID: \"a8b5311c-e2ce-4717-9009-8e53b9488740\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:41.047859 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:41.047815    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-03bf3578-g4zm2\" (UID: \"a8b5311c-e2ce-4717-9009-8e53b9488740\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:41.047982 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:41.047874    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-xks-ca\") pod \"maas-api-revoke-sub-03bf3578-g4zm2\" (UID: \"a8b5311c-e2ce-4717-9009-8e53b9488740\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:41.048654 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:41.048624    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-03bf3578-g4zm2\" (UID: \"a8b5311c-e2ce-4717-9009-8e53b9488740\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:41.050434 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:41.050410    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-xks-ca\") pod \"maas-api-revoke-sub-03bf3578-g4zm2\" (UID: \"a8b5311c-e2ce-4717-9009-8e53b9488740\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:41.061280 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:41.061247    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcwln\" (UniqueName: \"kubernetes.io/projected/a8b5311c-e2ce-4717-9009-8e53b9488740-kube-api-access-lcwln\") pod \"maas-api-revoke-sub-03bf3578-g4zm2\" (UID: \"a8b5311c-e2ce-4717-9009-8e53b9488740\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:41.147854 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:41.147769    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:41.439119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:41.439088    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"]
Nov 04 07:40:41.439815 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:40:41.439790    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b5311c_e2ce_4717_9009_8e53b9488740.slice/crio-04200f609df9e3e1acd40069b29c007abafd715af26ede1377041e76b312f72c WatchSource:0}: Error finding container 04200f609df9e3e1acd40069b29c007abafd715af26ede1377041e76b312f72c: Status 404 returned error can't find the container with id 04200f609df9e3e1acd40069b29c007abafd715af26ede1377041e76b312f72c
Nov 04 07:40:42.076399 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.076339    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2" event={"ID":"a8b5311c-e2ce-4717-9009-8e53b9488740","Type":"ContainerStarted","Data":"796ea98478c3ed6af263c33e0b6023ff99468275f2808f271deb2a6d5e053076"}
Nov 04 07:40:42.076607 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.076423    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2" event={"ID":"a8b5311c-e2ce-4717-9009-8e53b9488740","Type":"ContainerStarted","Data":"04200f609df9e3e1acd40069b29c007abafd715af26ede1377041e76b312f72c"}
Nov 04 07:40:42.100145 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.100084    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2" podStartSLOduration=2.10006564 podStartE2EDuration="2.10006564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:40:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:40:42.095677237 +0000 UTC m=+1464.117892497" watchObservedRunningTime="2026-11-04 07:40:42.10006564 +0000 UTC m=+1464.122280905"
Nov 04 07:40:42.664348 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.664295    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8" podUID="6004603c-c43c-41ea-9434-82954cb86dbe" containerName="payload-processing" containerID="cri-o://5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905" gracePeriod=25
Nov 04 07:40:42.955305 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.955278    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:42.987660 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.987620    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-provider-credentials\" (UniqueName: \"kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-provider-credentials\") pod \"6004603c-c43c-41ea-9434-82954cb86dbe\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") "
Nov 04 07:40:42.987883 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.987729    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-plugins-config-volume\") pod \"6004603c-c43c-41ea-9434-82954cb86dbe\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") "
Nov 04 07:40:42.987883 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.987839    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-routing-overlay-volume\") pod \"6004603c-c43c-41ea-9434-82954cb86dbe\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") "
Nov 04 07:40:42.988025 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.987945    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-kube-api-access-b9bwz\" (UniqueName: \"kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-kube-api-access-b9bwz\") pod \"6004603c-c43c-41ea-9434-82954cb86dbe\" (UID: \"6004603c-c43c-41ea-9434-82954cb86dbe\") "
Nov 04 07:40:42.988467 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.988203    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-routing-overlay-volume" pod "6004603c-c43c-41ea-9434-82954cb86dbe" (UID: "6004603c-c43c-41ea-9434-82954cb86dbe"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:40:42.988467 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.988219    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-plugins-config-volume" pod "6004603c-c43c-41ea-9434-82954cb86dbe" (UID: "6004603c-c43c-41ea-9434-82954cb86dbe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:40:42.988648 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.988546    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:42.988648 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.988569    5626 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6004603c-c43c-41ea-9434-82954cb86dbe-routing-overlay-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:42.990410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.990347    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-kube-api-access-b9bwz" pod "6004603c-c43c-41ea-9434-82954cb86dbe" (UID: "6004603c-c43c-41ea-9434-82954cb86dbe"). InnerVolumeSpecName "kube-api-access-b9bwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:40:42.990530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:42.990502    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-provider-credentials" pod "6004603c-c43c-41ea-9434-82954cb86dbe" (UID: "6004603c-c43c-41ea-9434-82954cb86dbe"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:40:43.087954 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.087915    5626 generic.go:356] "Generic (PLEG): container finished" podID="6004603c-c43c-41ea-9434-82954cb86dbe" containerID="5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905" exitCode=0
Nov 04 07:40:43.088154 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.087955    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8" event={"ID":"6004603c-c43c-41ea-9434-82954cb86dbe","Type":"ContainerDied","Data":"5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905"}
Nov 04 07:40:43.088154 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.088025    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8" event={"ID":"6004603c-c43c-41ea-9434-82954cb86dbe","Type":"ContainerDied","Data":"b72dd530638f4c253f6c3b2aaec44f0ce7d266e8fab387002a3ee22b3bae1bac"}
Nov 04 07:40:43.088154 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.088053    5626 scope.go:122] "RemoveContainer" containerID="5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905"
Nov 04 07:40:43.088489 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.088399    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"
Nov 04 07:40:43.089456 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.089373    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9bwz\" (UniqueName: \"kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-kube-api-access-b9bwz\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:43.089456 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.089391    5626 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/6004603c-c43c-41ea-9434-82954cb86dbe-provider-credentials\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:43.090333 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.090308    5626 generic.go:356] "Generic (PLEG): container finished" podID="a8b5311c-e2ce-4717-9009-8e53b9488740" containerID="796ea98478c3ed6af263c33e0b6023ff99468275f2808f271deb2a6d5e053076" exitCode=0
Nov 04 07:40:43.090493 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.090397    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2" event={"ID":"a8b5311c-e2ce-4717-9009-8e53b9488740","Type":"ContainerDied","Data":"796ea98478c3ed6af263c33e0b6023ff99468275f2808f271deb2a6d5e053076"}
Nov 04 07:40:43.099999 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.099978    5626 scope.go:122] "RemoveContainer" containerID="5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905"
Nov 04 07:40:43.100329 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:40:43.100310    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905\": container with ID starting with 5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905 not found: ID does not exist" containerID="5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905"
Nov 04 07:40:43.100415 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.100339    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905"} err="failed to get container status \"5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905\": rpc error: code = NotFound desc = could not find container \"5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905\": container with ID starting with 5726eddf5d542f348c98d0863d23665758d41d68b479a3f4d138e13ec2a1b905 not found: ID does not exist"
Nov 04 07:40:43.128203 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.128146    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"]
Nov 04 07:40:43.132498 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:43.132472    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8"]
Nov 04 07:40:44.096256 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:44.096218    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6004603c-c43c-41ea-9434-82954cb86dbe" path="/var/lib/kubelet/pods/6004603c-c43c-41ea-9434-82954cb86dbe/volumes"
Nov 04 07:40:44.264536 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:44.264497    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:44.305226 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:44.305180    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-openshift-ca\") pod \"a8b5311c-e2ce-4717-9009-8e53b9488740\" (UID: \"a8b5311c-e2ce-4717-9009-8e53b9488740\") "
Nov 04 07:40:44.305434 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:44.305315    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8b5311c-e2ce-4717-9009-8e53b9488740-kube-api-access-lcwln\" (UniqueName: \"kubernetes.io/projected/a8b5311c-e2ce-4717-9009-8e53b9488740-kube-api-access-lcwln\") pod \"a8b5311c-e2ce-4717-9009-8e53b9488740\" (UID: \"a8b5311c-e2ce-4717-9009-8e53b9488740\") "
Nov 04 07:40:44.305497 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:44.305445    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-xks-ca\") pod \"a8b5311c-e2ce-4717-9009-8e53b9488740\" (UID: \"a8b5311c-e2ce-4717-9009-8e53b9488740\") "
Nov 04 07:40:44.305685 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:44.305658    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-openshift-ca" pod "a8b5311c-e2ce-4717-9009-8e53b9488740" (UID: "a8b5311c-e2ce-4717-9009-8e53b9488740"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:40:44.305892 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:44.305863    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:44.309786 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:44.309752    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-xks-ca" pod "a8b5311c-e2ce-4717-9009-8e53b9488740" (UID: "a8b5311c-e2ce-4717-9009-8e53b9488740"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:40:44.309902 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:44.309826    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b5311c-e2ce-4717-9009-8e53b9488740-kube-api-access-lcwln" pod "a8b5311c-e2ce-4717-9009-8e53b9488740" (UID: "a8b5311c-e2ce-4717-9009-8e53b9488740"). InnerVolumeSpecName "kube-api-access-lcwln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:40:44.407006 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:44.406930    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcwln\" (UniqueName: \"kubernetes.io/projected/a8b5311c-e2ce-4717-9009-8e53b9488740-kube-api-access-lcwln\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:44.407006 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:44.406960    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8b5311c-e2ce-4717-9009-8e53b9488740-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:45.107374 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:45.107327    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2"
Nov 04 07:40:45.107774 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:45.107339    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-03bf3578-g4zm2" event={"ID":"a8b5311c-e2ce-4717-9009-8e53b9488740","Type":"ContainerDied","Data":"04200f609df9e3e1acd40069b29c007abafd715af26ede1377041e76b312f72c"}
Nov 04 07:40:45.107774 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:45.107428    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04200f609df9e3e1acd40069b29c007abafd715af26ede1377041e76b312f72c"
Nov 04 07:40:47.664229 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.664181    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"]
Nov 04 07:40:47.665216 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.665192    5626 cpu_manager.go:414] "containerMap: removing container" podUID="a8b5311c-e2ce-4717-9009-8e53b9488740" containerName="revoke-keys"
Nov 04 07:40:47.665315 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.665217    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8b5311c-e2ce-4717-9009-8e53b9488740" containerName="revoke-keys"
Nov 04 07:40:47.665315 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.665244    5626 cpu_manager.go:414] "containerMap: removing container" podUID="6004603c-c43c-41ea-9434-82954cb86dbe" containerName="payload-processing"
Nov 04 07:40:47.665315 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.665252    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6004603c-c43c-41ea-9434-82954cb86dbe" containerName="payload-processing"
Nov 04 07:40:47.665471 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.665454    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8b5311c-e2ce-4717-9009-8e53b9488740" containerName="revoke-keys"
Nov 04 07:40:47.665521 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.665476    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="6004603c-c43c-41ea-9434-82954cb86dbe" containerName="payload-processing"
Nov 04 07:40:47.669432 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.669414    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:47.676206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.676179    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:40:47.677061 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.677035    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:40:47.677185 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.677133    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:40:47.677248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.677199    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:40:47.678712 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.678683    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"]
Nov 04 07:40:47.750154 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.750119    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-xks-ca\") pod \"maas-api-revoke-sub-19f7830f-vwxqb\" (UID: \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:47.750329 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.750173    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pbqw\" (UniqueName: \"kubernetes.io/projected/a5e69a0d-c6ca-48e1-988f-76fca35a1183-kube-api-access-2pbqw\") pod \"maas-api-revoke-sub-19f7830f-vwxqb\" (UID: \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:47.750329 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.750247    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-19f7830f-vwxqb\" (UID: \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:47.852018 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.851959    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-xks-ca\") pod \"maas-api-revoke-sub-19f7830f-vwxqb\" (UID: \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:47.852228 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.852110    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pbqw\" (UniqueName: \"kubernetes.io/projected/a5e69a0d-c6ca-48e1-988f-76fca35a1183-kube-api-access-2pbqw\") pod \"maas-api-revoke-sub-19f7830f-vwxqb\" (UID: \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:47.852228 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.852149    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-19f7830f-vwxqb\" (UID: \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:47.852913 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.852889    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-19f7830f-vwxqb\" (UID: \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:47.854577 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.854554    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-xks-ca\") pod \"maas-api-revoke-sub-19f7830f-vwxqb\" (UID: \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:47.864985 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.864961    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pbqw\" (UniqueName: \"kubernetes.io/projected/a5e69a0d-c6ca-48e1-988f-76fca35a1183-kube-api-access-2pbqw\") pod \"maas-api-revoke-sub-19f7830f-vwxqb\" (UID: \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:47.991350 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:47.991264    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:48.133325 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.133287    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"]
Nov 04 07:40:48.139071 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.139051    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:48.149555 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.149532    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"]
Nov 04 07:40:48.257119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.257040    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8lhr\" (UniqueName: \"kubernetes.io/projected/4d1b8f8c-2e42-4274-b95d-e5c682e92079-kube-api-access-j8lhr\") pod \"maas-api-revoke-sub-2883a0a1-xqt6h\" (UID: \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:48.257119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.257095    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2883a0a1-xqt6h\" (UID: \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:48.257308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.257201    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2883a0a1-xqt6h\" (UID: \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:48.284551 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:40:48.284520    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5e69a0d_c6ca_48e1_988f_76fca35a1183.slice/crio-d2c8ad1281c2c702c7b8ab0d49e115c807efe161103785022757f6f5dfaba483 WatchSource:0}: Error finding container d2c8ad1281c2c702c7b8ab0d49e115c807efe161103785022757f6f5dfaba483: Status 404 returned error can't find the container with id d2c8ad1281c2c702c7b8ab0d49e115c807efe161103785022757f6f5dfaba483
Nov 04 07:40:48.286768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.286741    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"]
Nov 04 07:40:48.358048 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.358014    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8lhr\" (UniqueName: \"kubernetes.io/projected/4d1b8f8c-2e42-4274-b95d-e5c682e92079-kube-api-access-j8lhr\") pod \"maas-api-revoke-sub-2883a0a1-xqt6h\" (UID: \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:48.358225 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.358074    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2883a0a1-xqt6h\" (UID: \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:48.358225 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.358110    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2883a0a1-xqt6h\" (UID: \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:48.358820 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.358800    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2883a0a1-xqt6h\" (UID: \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:48.360349 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.360327    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2883a0a1-xqt6h\" (UID: \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:48.381277 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.381252    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8lhr\" (UniqueName: \"kubernetes.io/projected/4d1b8f8c-2e42-4274-b95d-e5c682e92079-kube-api-access-j8lhr\") pod \"maas-api-revoke-sub-2883a0a1-xqt6h\" (UID: \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:48.457119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.457091    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:48.762317 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:48.762286    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"]
Nov 04 07:40:48.764257 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:40:48.764227    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d1b8f8c_2e42_4274_b95d_e5c682e92079.slice/crio-54e96df0755da32e3ab7a832616a4280d4ad668c64c83228796a17e589109f9a WatchSource:0}: Error finding container 54e96df0755da32e3ab7a832616a4280d4ad668c64c83228796a17e589109f9a: Status 404 returned error can't find the container with id 54e96df0755da32e3ab7a832616a4280d4ad668c64c83228796a17e589109f9a
Nov 04 07:40:49.141425 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:49.141356    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h" event={"ID":"4d1b8f8c-2e42-4274-b95d-e5c682e92079","Type":"ContainerStarted","Data":"54e96df0755da32e3ab7a832616a4280d4ad668c64c83228796a17e589109f9a"}
Nov 04 07:40:49.144713 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:49.144685    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb" event={"ID":"a5e69a0d-c6ca-48e1-988f-76fca35a1183","Type":"ContainerStarted","Data":"1a1834268e6f10a605ce96392405fd14a81654e963702bb86b44b8088a41c99e"}
Nov 04 07:40:49.144834 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:49.144739    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb" event={"ID":"a5e69a0d-c6ca-48e1-988f-76fca35a1183","Type":"ContainerStarted","Data":"d2c8ad1281c2c702c7b8ab0d49e115c807efe161103785022757f6f5dfaba483"}
Nov 04 07:40:49.180864 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:49.180779    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb" podStartSLOduration=2.180764874 podStartE2EDuration="2.180764874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:40:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:40:49.179713511 +0000 UTC m=+1471.201928770" watchObservedRunningTime="2026-11-04 07:40:49.180764874 +0000 UTC m=+1471.202980135"
Nov 04 07:40:50.153175 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:50.153140    5626 generic.go:356] "Generic (PLEG): container finished" podID="4d1b8f8c-2e42-4274-b95d-e5c682e92079" containerID="346f3711cd66824fbe555ed90ebfa1bd8f880a3f47dc376a981d7a431f8f9ab8" exitCode=0
Nov 04 07:40:50.153640 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:50.153228    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h" event={"ID":"4d1b8f8c-2e42-4274-b95d-e5c682e92079","Type":"ContainerDied","Data":"346f3711cd66824fbe555ed90ebfa1bd8f880a3f47dc376a981d7a431f8f9ab8"}
Nov 04 07:40:50.154915 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:50.154893    5626 generic.go:356] "Generic (PLEG): container finished" podID="a5e69a0d-c6ca-48e1-988f-76fca35a1183" containerID="1a1834268e6f10a605ce96392405fd14a81654e963702bb86b44b8088a41c99e" exitCode=0
Nov 04 07:40:50.155022 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:50.154918    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb" event={"ID":"a5e69a0d-c6ca-48e1-988f-76fca35a1183","Type":"ContainerDied","Data":"1a1834268e6f10a605ce96392405fd14a81654e963702bb86b44b8088a41c99e"}
Nov 04 07:40:51.341005 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.339987    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:51.375551 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.375526    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:51.404296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.401614    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-xks-ca\") pod \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\" (UID: \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\") "
Nov 04 07:40:51.404296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.401742    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d1b8f8c-2e42-4274-b95d-e5c682e92079-kube-api-access-j8lhr\" (UniqueName: \"kubernetes.io/projected/4d1b8f8c-2e42-4274-b95d-e5c682e92079-kube-api-access-j8lhr\") pod \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\" (UID: \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\") "
Nov 04 07:40:51.404296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.401825    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-openshift-ca\") pod \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\" (UID: \"4d1b8f8c-2e42-4274-b95d-e5c682e92079\") "
Nov 04 07:40:51.404296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.404247    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-openshift-ca" pod "4d1b8f8c-2e42-4274-b95d-e5c682e92079" (UID: "4d1b8f8c-2e42-4274-b95d-e5c682e92079"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:40:51.404910 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.404881    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-xks-ca" pod "4d1b8f8c-2e42-4274-b95d-e5c682e92079" (UID: "4d1b8f8c-2e42-4274-b95d-e5c682e92079"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:40:51.405032 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.405009    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d1b8f8c-2e42-4274-b95d-e5c682e92079-kube-api-access-j8lhr" pod "4d1b8f8c-2e42-4274-b95d-e5c682e92079" (UID: "4d1b8f8c-2e42-4274-b95d-e5c682e92079"). InnerVolumeSpecName "kube-api-access-j8lhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:40:51.503000 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.502967    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5e69a0d-c6ca-48e1-988f-76fca35a1183-kube-api-access-2pbqw\" (UniqueName: \"kubernetes.io/projected/a5e69a0d-c6ca-48e1-988f-76fca35a1183-kube-api-access-2pbqw\") pod \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\" (UID: \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\") "
Nov 04 07:40:51.503195 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.503182    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-xks-ca\") pod \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\" (UID: \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\") "
Nov 04 07:40:51.503258 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.503219    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-openshift-ca\") pod \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\" (UID: \"a5e69a0d-c6ca-48e1-988f-76fca35a1183\") "
Nov 04 07:40:51.503831 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.503762    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-openshift-ca" pod "a5e69a0d-c6ca-48e1-988f-76fca35a1183" (UID: "a5e69a0d-c6ca-48e1-988f-76fca35a1183"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:40:51.503831 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.503825    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:51.504019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.503842    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8lhr\" (UniqueName: \"kubernetes.io/projected/4d1b8f8c-2e42-4274-b95d-e5c682e92079-kube-api-access-j8lhr\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:51.504019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.503863    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4d1b8f8c-2e42-4274-b95d-e5c682e92079-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:51.505282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.505255    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5e69a0d-c6ca-48e1-988f-76fca35a1183-kube-api-access-2pbqw" pod "a5e69a0d-c6ca-48e1-988f-76fca35a1183" (UID: "a5e69a0d-c6ca-48e1-988f-76fca35a1183"). InnerVolumeSpecName "kube-api-access-2pbqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:40:51.505398 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.505305    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-xks-ca" pod "a5e69a0d-c6ca-48e1-988f-76fca35a1183" (UID: "a5e69a0d-c6ca-48e1-988f-76fca35a1183"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:40:51.604899 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.604852    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:51.604899 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.604892    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5e69a0d-c6ca-48e1-988f-76fca35a1183-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:51.605154 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.604908    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pbqw\" (UniqueName: \"kubernetes.io/projected/a5e69a0d-c6ca-48e1-988f-76fca35a1183-kube-api-access-2pbqw\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:51.912842 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:51.912773    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"]
Nov 04 07:40:52.269940 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:52.269854    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"]
Nov 04 07:40:52.275060 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:52.275031    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h"
Nov 04 07:40:52.275211 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:52.275031    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2883a0a1-xqt6h" event={"ID":"4d1b8f8c-2e42-4274-b95d-e5c682e92079","Type":"ContainerDied","Data":"54e96df0755da32e3ab7a832616a4280d4ad668c64c83228796a17e589109f9a"}
Nov 04 07:40:52.275211 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:52.275157    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54e96df0755da32e3ab7a832616a4280d4ad668c64c83228796a17e589109f9a"
Nov 04 07:40:52.276798 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:52.276778    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb"
Nov 04 07:40:52.276927 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:52.276903    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-19f7830f-vwxqb" event={"ID":"a5e69a0d-c6ca-48e1-988f-76fca35a1183","Type":"ContainerDied","Data":"d2c8ad1281c2c702c7b8ab0d49e115c807efe161103785022757f6f5dfaba483"}
Nov 04 07:40:52.276987 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:52.276945    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2c8ad1281c2c702c7b8ab0d49e115c807efe161103785022757f6f5dfaba483"
Nov 04 07:40:56.913940 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:56.913896    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6" podUID="181b7abe-885a-47f4-97d3-0b40fc4bf994" containerName="payload-processing" containerID="cri-o://3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62" gracePeriod=25
Nov 04 07:40:57.189339 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.189308    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:40:57.282586 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.282555    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-routing-overlay-volume\") pod \"181b7abe-885a-47f4-97d3-0b40fc4bf994\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") "
Nov 04 07:40:57.282760 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.282617    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-provider-credentials\" (UniqueName: \"kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-provider-credentials\") pod \"181b7abe-885a-47f4-97d3-0b40fc4bf994\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") "
Nov 04 07:40:57.282818 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.282802    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-plugins-config-volume\") pod \"181b7abe-885a-47f4-97d3-0b40fc4bf994\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") "
Nov 04 07:40:57.282923 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.282895    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-kube-api-access-s4l4b\" (UniqueName: \"kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-kube-api-access-s4l4b\") pod \"181b7abe-885a-47f4-97d3-0b40fc4bf994\" (UID: \"181b7abe-885a-47f4-97d3-0b40fc4bf994\") "
Nov 04 07:40:57.283104 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.283072    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-routing-overlay-volume" pod "181b7abe-885a-47f4-97d3-0b40fc4bf994" (UID: "181b7abe-885a-47f4-97d3-0b40fc4bf994"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:40:57.283252 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.283228    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-plugins-config-volume" pod "181b7abe-885a-47f4-97d3-0b40fc4bf994" (UID: "181b7abe-885a-47f4-97d3-0b40fc4bf994"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:40:57.283478 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.283459    5626 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-routing-overlay-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:57.283478 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.283478    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/181b7abe-885a-47f4-97d3-0b40fc4bf994-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:57.285076 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.285049    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-kube-api-access-s4l4b" pod "181b7abe-885a-47f4-97d3-0b40fc4bf994" (UID: "181b7abe-885a-47f4-97d3-0b40fc4bf994"). InnerVolumeSpecName "kube-api-access-s4l4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:40:57.285153 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.285084    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-provider-credentials" pod "181b7abe-885a-47f4-97d3-0b40fc4bf994" (UID: "181b7abe-885a-47f4-97d3-0b40fc4bf994"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:40:57.313812 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.313777    5626 generic.go:356] "Generic (PLEG): container finished" podID="181b7abe-885a-47f4-97d3-0b40fc4bf994" containerID="3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62" exitCode=0
Nov 04 07:40:57.313929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.313823    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"
Nov 04 07:40:57.313929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.313859    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6" event={"ID":"181b7abe-885a-47f4-97d3-0b40fc4bf994","Type":"ContainerDied","Data":"3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62"}
Nov 04 07:40:57.313929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.313914    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6" event={"ID":"181b7abe-885a-47f4-97d3-0b40fc4bf994","Type":"ContainerDied","Data":"fa7075d537df93b85cea3eca4cc2a37e0ef15c5ec6c75d726913bf4a6f519165"}
Nov 04 07:40:57.314038 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.313936    5626 scope.go:122] "RemoveContainer" containerID="3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62"
Nov 04 07:40:57.324995 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.324975    5626 scope.go:122] "RemoveContainer" containerID="3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62"
Nov 04 07:40:57.325247 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:40:57.325228    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62\": container with ID starting with 3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62 not found: ID does not exist" containerID="3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62"
Nov 04 07:40:57.325311 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.325256    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62"} err="failed to get container status \"3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62\": rpc error: code = NotFound desc = could not find container \"3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62\": container with ID starting with 3a918ce84d877a9d671fe105b6322623175781c355c8a173be6338a981c66d62 not found: ID does not exist"
Nov 04 07:40:57.339096 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.339074    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"]
Nov 04 07:40:57.346151 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.346125    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-a558a6/payload-processing-external-model-e2e-extmod-b-a558a6-787dmf6r6"]
Nov 04 07:40:57.385211 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.385177    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4l4b\" (UniqueName: \"kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-kube-api-access-s4l4b\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:57.385211 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:57.385208    5626 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/181b7abe-885a-47f4-97d3-0b40fc4bf994-provider-credentials\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:40:58.095278 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:58.095244    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="181b7abe-885a-47f4-97d3-0b40fc4bf994" path="/var/lib/kubelet/pods/181b7abe-885a-47f4-97d3-0b40fc4bf994/volumes"
Nov 04 07:40:58.695706 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:58.695673    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"]
Nov 04 07:40:58.702537 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:58.702493    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"]
Nov 04 07:40:59.025270 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.019697    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"]
Nov 04 07:40:59.025270 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.020759    5626 cpu_manager.go:414] "containerMap: removing container" podUID="a5e69a0d-c6ca-48e1-988f-76fca35a1183" containerName="revoke-keys"
Nov 04 07:40:59.025270 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.020780    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5e69a0d-c6ca-48e1-988f-76fca35a1183" containerName="revoke-keys"
Nov 04 07:40:59.025270 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.020817    5626 cpu_manager.go:414] "containerMap: removing container" podUID="4d1b8f8c-2e42-4274-b95d-e5c682e92079" containerName="revoke-keys"
Nov 04 07:40:59.025270 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.020826    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d1b8f8c-2e42-4274-b95d-e5c682e92079" containerName="revoke-keys"
Nov 04 07:40:59.025270 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.020871    5626 cpu_manager.go:414] "containerMap: removing container" podUID="181b7abe-885a-47f4-97d3-0b40fc4bf994" containerName="payload-processing"
Nov 04 07:40:59.025270 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.020879    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="181b7abe-885a-47f4-97d3-0b40fc4bf994" containerName="payload-processing"
Nov 04 07:40:59.025270 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.021053    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="181b7abe-885a-47f4-97d3-0b40fc4bf994" containerName="payload-processing"
Nov 04 07:40:59.025270 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.021071    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5e69a0d-c6ca-48e1-988f-76fca35a1183" containerName="revoke-keys"
Nov 04 07:40:59.025270 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.021085    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d1b8f8c-2e42-4274-b95d-e5c682e92079" containerName="revoke-keys"
Nov 04 07:40:59.027879 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.027518    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:40:59.032831 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.032759    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:40:59.032979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.032872    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:40:59.033327 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.033302    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:40:59.033522 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.033405    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:40:59.050388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.045626    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"]
Nov 04 07:40:59.119248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.119201    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0e9be1a1-5ndst\" (UID: \"d4be153f-9b45-4b53-acea-1b2bfac28202\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:40:59.119767 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.119308    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v78cn\" (UniqueName: \"kubernetes.io/projected/d4be153f-9b45-4b53-acea-1b2bfac28202-kube-api-access-v78cn\") pod \"maas-api-revoke-sub-0e9be1a1-5ndst\" (UID: \"d4be153f-9b45-4b53-acea-1b2bfac28202\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:40:59.119767 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.119539    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0e9be1a1-5ndst\" (UID: \"d4be153f-9b45-4b53-acea-1b2bfac28202\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:40:59.220351 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.220308    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0e9be1a1-5ndst\" (UID: \"d4be153f-9b45-4b53-acea-1b2bfac28202\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:40:59.220578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.220393    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v78cn\" (UniqueName: \"kubernetes.io/projected/d4be153f-9b45-4b53-acea-1b2bfac28202-kube-api-access-v78cn\") pod \"maas-api-revoke-sub-0e9be1a1-5ndst\" (UID: \"d4be153f-9b45-4b53-acea-1b2bfac28202\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:40:59.220578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.220472    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0e9be1a1-5ndst\" (UID: \"d4be153f-9b45-4b53-acea-1b2bfac28202\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:40:59.221192 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.221169    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0e9be1a1-5ndst\" (UID: \"d4be153f-9b45-4b53-acea-1b2bfac28202\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:40:59.222803 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.222778    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0e9be1a1-5ndst\" (UID: \"d4be153f-9b45-4b53-acea-1b2bfac28202\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:40:59.234654 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.234607    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v78cn\" (UniqueName: \"kubernetes.io/projected/d4be153f-9b45-4b53-acea-1b2bfac28202-kube-api-access-v78cn\") pod \"maas-api-revoke-sub-0e9be1a1-5ndst\" (UID: \"d4be153f-9b45-4b53-acea-1b2bfac28202\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:40:59.361308 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.361279    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:40:59.664504 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:40:59.664464    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4be153f_9b45_4b53_acea_1b2bfac28202.slice/crio-1f173f8761d36ff1fdb3045d182539612c895706df1c9eab14aca7e50e05f6fe WatchSource:0}: Error finding container 1f173f8761d36ff1fdb3045d182539612c895706df1c9eab14aca7e50e05f6fe: Status 404 returned error can't find the container with id 1f173f8761d36ff1fdb3045d182539612c895706df1c9eab14aca7e50e05f6fe
Nov 04 07:40:59.665535 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:40:59.665504    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"]
Nov 04 07:41:00.439956 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.438798    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"]
Nov 04 07:41:00.458503 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.458475    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"]
Nov 04 07:41:00.458735 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.458717    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst" event={"ID":"d4be153f-9b45-4b53-acea-1b2bfac28202","Type":"ContainerStarted","Data":"35e1f092c00ef2d079bdcfd8587d57fc4df1db90b59c1d3447f00f4e934102aa"}
Nov 04 07:41:00.458853 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.458839    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst" event={"ID":"d4be153f-9b45-4b53-acea-1b2bfac28202","Type":"ContainerStarted","Data":"1f173f8761d36ff1fdb3045d182539612c895706df1c9eab14aca7e50e05f6fe"}
Nov 04 07:41:00.459024 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.459011    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.477025 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.476978    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst" podStartSLOduration=2.476964472 podStartE2EDuration="2.476964472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:40:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:41:00.474628404 +0000 UTC m=+1482.496843664" watchObservedRunningTime="2026-11-04 07:41:00.476964472 +0000 UTC m=+1482.499179732"
Nov 04 07:41:00.557906 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.557871    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-plugins-config-volume\") pod \"payload-processing-external-model-8474b49dbc-dvj9v\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.557906 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.557903    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qch4h\" (UniqueName: \"kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-kube-api-access-qch4h\") pod \"payload-processing-external-model-8474b49dbc-dvj9v\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.558163 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.558024    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-provider-credentials\") pod \"payload-processing-external-model-8474b49dbc-dvj9v\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.558163 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.558060    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-routing-overlay-volume\") pod \"payload-processing-external-model-8474b49dbc-dvj9v\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.659136 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.659101    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-plugins-config-volume\") pod \"payload-processing-external-model-8474b49dbc-dvj9v\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.659136 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.659138    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qch4h\" (UniqueName: \"kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-kube-api-access-qch4h\") pod \"payload-processing-external-model-8474b49dbc-dvj9v\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.659374 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.659168    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-provider-credentials\") pod \"payload-processing-external-model-8474b49dbc-dvj9v\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.659374 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.659204    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-routing-overlay-volume\") pod \"payload-processing-external-model-8474b49dbc-dvj9v\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.659897 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.659873    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-plugins-config-volume\") pod \"payload-processing-external-model-8474b49dbc-dvj9v\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.660010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.659893    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-routing-overlay-volume\") pod \"payload-processing-external-model-8474b49dbc-dvj9v\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.661686 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.661650    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-provider-credentials\") pod \"payload-processing-external-model-8474b49dbc-dvj9v\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.672156 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.672123    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qch4h\" (UniqueName: \"kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-kube-api-access-qch4h\") pod \"payload-processing-external-model-8474b49dbc-dvj9v\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") " pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:00.777011 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:00.776978    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:01.067060 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:41:01.067014    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1697d471_a2ca_4151_8a7a_16b50c6d3b2b.slice/crio-79dac47e0826e9ba2a3652ea8c0f8b072b2ee74d2be47832be1b8ca2c8670928 WatchSource:0}: Error finding container 79dac47e0826e9ba2a3652ea8c0f8b072b2ee74d2be47832be1b8ca2c8670928: Status 404 returned error can't find the container with id 79dac47e0826e9ba2a3652ea8c0f8b072b2ee74d2be47832be1b8ca2c8670928
Nov 04 07:41:01.069047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:01.069023    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"]
Nov 04 07:41:01.483152 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:01.483058    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v" event={"ID":"1697d471-a2ca-4151-8a7a-16b50c6d3b2b","Type":"ContainerStarted","Data":"34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9"}
Nov 04 07:41:01.483666 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:01.483155    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:01.483666 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:01.483184    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v" event={"ID":"1697d471-a2ca-4151-8a7a-16b50c6d3b2b","Type":"ContainerStarted","Data":"79dac47e0826e9ba2a3652ea8c0f8b072b2ee74d2be47832be1b8ca2c8670928"}
Nov 04 07:41:01.484937 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:01.484907    5626 generic.go:356] "Generic (PLEG): container finished" podID="d4be153f-9b45-4b53-acea-1b2bfac28202" containerID="35e1f092c00ef2d079bdcfd8587d57fc4df1db90b59c1d3447f00f4e934102aa" exitCode=0
Nov 04 07:41:01.485071 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:01.484993    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst" event={"ID":"d4be153f-9b45-4b53-acea-1b2bfac28202","Type":"ContainerDied","Data":"35e1f092c00ef2d079bdcfd8587d57fc4df1db90b59c1d3447f00f4e934102aa"}
Nov 04 07:41:01.507578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:01.507527    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v" podStartSLOduration=1.507511941 podStartE2EDuration="1.507511941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:41:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:41:01.502443874 +0000 UTC m=+1483.524659134" watchObservedRunningTime="2026-11-04 07:41:01.507511941 +0000 UTC m=+1483.529727201"
Nov 04 07:41:02.747567 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:02.743983    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:41:02.801060 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:02.799703    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-openshift-ca\") pod \"d4be153f-9b45-4b53-acea-1b2bfac28202\" (UID: \"d4be153f-9b45-4b53-acea-1b2bfac28202\") "
Nov 04 07:41:02.801060 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:02.799818    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4be153f-9b45-4b53-acea-1b2bfac28202-kube-api-access-v78cn\" (UniqueName: \"kubernetes.io/projected/d4be153f-9b45-4b53-acea-1b2bfac28202-kube-api-access-v78cn\") pod \"d4be153f-9b45-4b53-acea-1b2bfac28202\" (UID: \"d4be153f-9b45-4b53-acea-1b2bfac28202\") "
Nov 04 07:41:02.801060 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:02.800013    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-xks-ca\") pod \"d4be153f-9b45-4b53-acea-1b2bfac28202\" (UID: \"d4be153f-9b45-4b53-acea-1b2bfac28202\") "
Nov 04 07:41:02.804716 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:02.802396    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-openshift-ca" pod "d4be153f-9b45-4b53-acea-1b2bfac28202" (UID: "d4be153f-9b45-4b53-acea-1b2bfac28202"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:02.812974 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:02.812935    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4be153f-9b45-4b53-acea-1b2bfac28202-kube-api-access-v78cn" pod "d4be153f-9b45-4b53-acea-1b2bfac28202" (UID: "d4be153f-9b45-4b53-acea-1b2bfac28202"). InnerVolumeSpecName "kube-api-access-v78cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:02.815808 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:02.815771    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-xks-ca" pod "d4be153f-9b45-4b53-acea-1b2bfac28202" (UID: "d4be153f-9b45-4b53-acea-1b2bfac28202"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:41:02.903845 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:02.903808    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:02.903845 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:02.903854    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v78cn\" (UniqueName: \"kubernetes.io/projected/d4be153f-9b45-4b53-acea-1b2bfac28202-kube-api-access-v78cn\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:02.904119 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:02.903874    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4be153f-9b45-4b53-acea-1b2bfac28202-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:03.279386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.271349    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"]
Nov 04 07:41:03.279386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.272430    5626 cpu_manager.go:414] "containerMap: removing container" podUID="d4be153f-9b45-4b53-acea-1b2bfac28202" containerName="revoke-keys"
Nov 04 07:41:03.279386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.272452    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4be153f-9b45-4b53-acea-1b2bfac28202" containerName="revoke-keys"
Nov 04 07:41:03.279386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.272619    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4be153f-9b45-4b53-acea-1b2bfac28202" containerName="revoke-keys"
Nov 04 07:41:03.323155 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.323119    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"]
Nov 04 07:41:03.323327 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.323243    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:03.422229 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.421668    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h22xm\" (UniqueName: \"kubernetes.io/projected/4e15c935-b53a-4307-ac3b-66a07729f311-kube-api-access-h22xm\") pod \"maas-api-revoke-sub-1e59041f-cc5hn\" (UID: \"4e15c935-b53a-4307-ac3b-66a07729f311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:03.422229 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.421777    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1e59041f-cc5hn\" (UID: \"4e15c935-b53a-4307-ac3b-66a07729f311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:03.422229 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.421839    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1e59041f-cc5hn\" (UID: \"4e15c935-b53a-4307-ac3b-66a07729f311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:03.576444 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.566691    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h22xm\" (UniqueName: \"kubernetes.io/projected/4e15c935-b53a-4307-ac3b-66a07729f311-kube-api-access-h22xm\") pod \"maas-api-revoke-sub-1e59041f-cc5hn\" (UID: \"4e15c935-b53a-4307-ac3b-66a07729f311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:03.576444 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.566796    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1e59041f-cc5hn\" (UID: \"4e15c935-b53a-4307-ac3b-66a07729f311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:03.576444 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.566854    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1e59041f-cc5hn\" (UID: \"4e15c935-b53a-4307-ac3b-66a07729f311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:03.600385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.598334    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1e59041f-cc5hn\" (UID: \"4e15c935-b53a-4307-ac3b-66a07729f311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:03.637385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.636874    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1e59041f-cc5hn\" (UID: \"4e15c935-b53a-4307-ac3b-66a07729f311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:03.644385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.638098    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst" event={"ID":"d4be153f-9b45-4b53-acea-1b2bfac28202","Type":"ContainerDied","Data":"1f173f8761d36ff1fdb3045d182539612c895706df1c9eab14aca7e50e05f6fe"}
Nov 04 07:41:03.644385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.638164    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f173f8761d36ff1fdb3045d182539612c895706df1c9eab14aca7e50e05f6fe"
Nov 04 07:41:03.644385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.638234    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0e9be1a1-5ndst"
Nov 04 07:41:03.644385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.644052    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h22xm\" (UniqueName: \"kubernetes.io/projected/4e15c935-b53a-4307-ac3b-66a07729f311-kube-api-access-h22xm\") pod \"maas-api-revoke-sub-1e59041f-cc5hn\" (UID: \"4e15c935-b53a-4307-ac3b-66a07729f311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:03.701389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.698745    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5" podUID="6749a48e-433c-4073-9d11-1391830d9514" containerName="payload-processing" containerID="cri-o://f868fc4aaba50f8f85695c21b9775966d3c873f1b86893a4ad811de1b4d7ba0d" gracePeriod=25
Nov 04 07:41:03.712047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.704505    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg" podUID="e051d9eb-49d3-4277-8e05-a1ca78585d9f" containerName="payload-pre-processing" containerID="cri-o://dfac28bec78408d1dc88e407766a855d6b2f7e4c72a4bc8292a85dfd75926e8b" gracePeriod=25
Nov 04 07:41:03.754398 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:03.753604    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:04.990440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:04.989472    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:41:04.990440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:04.989546    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:41:05.382389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:05.381467    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:41:05.382389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:05.381535    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:41:05.389345 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:05.389172    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"]
Nov 04 07:41:05.509992 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:05.509959    5626 generic.go:356] "Generic (PLEG): container finished" podID="6749a48e-433c-4073-9d11-1391830d9514" containerID="f868fc4aaba50f8f85695c21b9775966d3c873f1b86893a4ad811de1b4d7ba0d" exitCode=0
Nov 04 07:41:05.510412 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:05.510384    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5" event={"ID":"6749a48e-433c-4073-9d11-1391830d9514","Type":"ContainerDied","Data":"f868fc4aaba50f8f85695c21b9775966d3c873f1b86893a4ad811de1b4d7ba0d"}
Nov 04 07:41:05.515640 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:05.515619    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"
Nov 04 07:41:05.616741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:05.616296    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6749a48e-433c-4073-9d11-1391830d9514-kube-api-access-snfgf\" (UniqueName: \"kubernetes.io/projected/6749a48e-433c-4073-9d11-1391830d9514-kube-api-access-snfgf\") pod \"6749a48e-433c-4073-9d11-1391830d9514\" (UID: \"6749a48e-433c-4073-9d11-1391830d9514\") "
Nov 04 07:41:05.616741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:05.616597    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6749a48e-433c-4073-9d11-1391830d9514-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6749a48e-433c-4073-9d11-1391830d9514-plugins-config-volume\") pod \"6749a48e-433c-4073-9d11-1391830d9514\" (UID: \"6749a48e-433c-4073-9d11-1391830d9514\") "
Nov 04 07:41:05.617562 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:05.617527    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6749a48e-433c-4073-9d11-1391830d9514-plugins-config-volume" pod "6749a48e-433c-4073-9d11-1391830d9514" (UID: "6749a48e-433c-4073-9d11-1391830d9514"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:05.617956 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:05.617925    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6749a48e-433c-4073-9d11-1391830d9514-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:05.621560 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:05.621533    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6749a48e-433c-4073-9d11-1391830d9514-kube-api-access-snfgf" pod "6749a48e-433c-4073-9d11-1391830d9514" (UID: "6749a48e-433c-4073-9d11-1391830d9514"). InnerVolumeSpecName "kube-api-access-snfgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:05.719695 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:05.719649    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snfgf\" (UniqueName: \"kubernetes.io/projected/6749a48e-433c-4073-9d11-1391830d9514-kube-api-access-snfgf\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:06.174013 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:06.173924    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:41:06.519038 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:06.518946    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn" event={"ID":"4e15c935-b53a-4307-ac3b-66a07729f311","Type":"ContainerStarted","Data":"c196091afb62397592efaa3ea386a6b8a429748f863775b8be9a9c17bfc09aed"}
Nov 04 07:41:06.519038 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:06.519002    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn" event={"ID":"4e15c935-b53a-4307-ac3b-66a07729f311","Type":"ContainerStarted","Data":"e57582471c015808c25c71d4b76157ef995da235919b213d698d6acfee3a67ea"}
Nov 04 07:41:06.520342 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:06.520324    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"
Nov 04 07:41:06.520466 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:06.520346    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5" event={"ID":"6749a48e-433c-4073-9d11-1391830d9514","Type":"ContainerDied","Data":"a9b59029ab00865c7e222368675d8baa00602d30a7cee70329e157e56f7ec268"}
Nov 04 07:41:06.520466 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:06.520405    5626 scope.go:122] "RemoveContainer" containerID="f868fc4aaba50f8f85695c21b9775966d3c873f1b86893a4ad811de1b4d7ba0d"
Nov 04 07:41:06.557413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:06.555265    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn" podStartSLOduration=3.5552458810000003 podStartE2EDuration="3.555245881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:41:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:41:06.552538129 +0000 UTC m=+1488.574753389" watchObservedRunningTime="2026-11-04 07:41:06.555245881 +0000 UTC m=+1488.577461140"
Nov 04 07:41:06.573530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:06.573491    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"]
Nov 04 07:41:06.579907 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:06.579879    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5"]
Nov 04 07:41:07.294988 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.294950    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main" containerID="cri-o://64a3e15b7acb57060e1809d5b00fceb29f218d07846023226c2789d90d29a00d" gracePeriod=45
Nov 04 07:41:07.530282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.530243    5626 generic.go:356] "Generic (PLEG): container finished" podID="4e15c935-b53a-4307-ac3b-66a07729f311" containerID="c196091afb62397592efaa3ea386a6b8a429748f863775b8be9a9c17bfc09aed" exitCode=0
Nov 04 07:41:07.530525 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.530401    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn" event={"ID":"4e15c935-b53a-4307-ac3b-66a07729f311","Type":"ContainerDied","Data":"c196091afb62397592efaa3ea386a6b8a429748f863775b8be9a9c17bfc09aed"}
Nov 04 07:41:07.533717 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.533685    5626 generic.go:356] "Generic (PLEG): container finished" podID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerID="64a3e15b7acb57060e1809d5b00fceb29f218d07846023226c2789d90d29a00d" exitCode=0
Nov 04 07:41:07.533851 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.533756    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" event={"ID":"44b06376-8c4a-45ac-a97e-a6e1cb982b32","Type":"ContainerDied","Data":"64a3e15b7acb57060e1809d5b00fceb29f218d07846023226c2789d90d29a00d"}
Nov 04 07:41:07.581644 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.581615    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:41:07.664672 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.664623    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-home\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-home\") pod \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") "
Nov 04 07:41:07.664841 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.664720    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44b06376-8c4a-45ac-a97e-a6e1cb982b32-kube-api-access-lfslg\" (UniqueName: \"kubernetes.io/projected/44b06376-8c4a-45ac-a97e-a6e1cb982b32-kube-api-access-lfslg\") pod \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") "
Nov 04 07:41:07.664920 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.664895    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tls-certs\" (UniqueName: \"kubernetes.io/secret/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tls-certs\") pod \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") "
Nov 04 07:41:07.664979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.664927    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-dshm\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-dshm\") pod \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") "
Nov 04 07:41:07.664979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.664942    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-home" pod "44b06376-8c4a-45ac-a97e-a6e1cb982b32" (UID: "44b06376-8c4a-45ac-a97e-a6e1cb982b32"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:07.664979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.664962    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-model-cache\") pod \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") "
Nov 04 07:41:07.665136 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.665038    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tmp-dir\") pod \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\" (UID: \"44b06376-8c4a-45ac-a97e-a6e1cb982b32\") "
Nov 04 07:41:07.665709 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.665355    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-model-cache" pod "44b06376-8c4a-45ac-a97e-a6e1cb982b32" (UID: "44b06376-8c4a-45ac-a97e-a6e1cb982b32"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:07.665709 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.665412    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tmp-dir" pod "44b06376-8c4a-45ac-a97e-a6e1cb982b32" (UID: "44b06376-8c4a-45ac-a97e-a6e1cb982b32"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:07.665709 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.665651    5626 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-home\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:07.665709 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.665672    5626 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-model-cache\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:07.665709 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.665687    5626 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tmp-dir\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:07.667945 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.667916    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44b06376-8c4a-45ac-a97e-a6e1cb982b32-kube-api-access-lfslg" pod "44b06376-8c4a-45ac-a97e-a6e1cb982b32" (UID: "44b06376-8c4a-45ac-a97e-a6e1cb982b32"). InnerVolumeSpecName "kube-api-access-lfslg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:07.668058 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.668039    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tls-certs" pod "44b06376-8c4a-45ac-a97e-a6e1cb982b32" (UID: "44b06376-8c4a-45ac-a97e-a6e1cb982b32"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:41:07.668123 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.668093    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-dshm" pod "44b06376-8c4a-45ac-a97e-a6e1cb982b32" (UID: "44b06376-8c4a-45ac-a97e-a6e1cb982b32"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:07.766860 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.766826    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfslg\" (UniqueName: \"kubernetes.io/projected/44b06376-8c4a-45ac-a97e-a6e1cb982b32-kube-api-access-lfslg\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:07.766860 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.766858    5626 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44b06376-8c4a-45ac-a97e-a6e1cb982b32-tls-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:07.767073 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:07.766870    5626 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44b06376-8c4a-45ac-a97e-a6e1cb982b32-dshm\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:08.095795 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.095763    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6749a48e-433c-4073-9d11-1391830d9514" path="/var/lib/kubelet/pods/6749a48e-433c-4073-9d11-1391830d9514/volumes"
Nov 04 07:41:08.542390 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.542335    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"
Nov 04 07:41:08.542886 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.542356    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc" event={"ID":"44b06376-8c4a-45ac-a97e-a6e1cb982b32","Type":"ContainerDied","Data":"692783ffbbab57b4a4256d37f06a9ea67169439f57571079b54b9c136ac55bf5"}
Nov 04 07:41:08.542886 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.542445    5626 scope.go:122] "RemoveContainer" containerID="64a3e15b7acb57060e1809d5b00fceb29f218d07846023226c2789d90d29a00d"
Nov 04 07:41:08.568058 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.568031    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"]
Nov 04 07:41:08.576912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.576881    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-29d3ee/simulator-w1-29d3ee-kserve-6cf8c87678-fjjjc"]
Nov 04 07:41:08.691298 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.691269    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:08.780721 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.780691    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-xks-ca\") pod \"4e15c935-b53a-4307-ac3b-66a07729f311\" (UID: \"4e15c935-b53a-4307-ac3b-66a07729f311\") "
Nov 04 07:41:08.780920 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.780739    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-openshift-ca\") pod \"4e15c935-b53a-4307-ac3b-66a07729f311\" (UID: \"4e15c935-b53a-4307-ac3b-66a07729f311\") "
Nov 04 07:41:08.780920 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.780901    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e15c935-b53a-4307-ac3b-66a07729f311-kube-api-access-h22xm\" (UniqueName: \"kubernetes.io/projected/4e15c935-b53a-4307-ac3b-66a07729f311-kube-api-access-h22xm\") pod \"4e15c935-b53a-4307-ac3b-66a07729f311\" (UID: \"4e15c935-b53a-4307-ac3b-66a07729f311\") "
Nov 04 07:41:08.781138 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.781112    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-openshift-ca" pod "4e15c935-b53a-4307-ac3b-66a07729f311" (UID: "4e15c935-b53a-4307-ac3b-66a07729f311"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:08.781297 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.781282    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:08.782898 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.782874    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-xks-ca" pod "4e15c935-b53a-4307-ac3b-66a07729f311" (UID: "4e15c935-b53a-4307-ac3b-66a07729f311"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:41:08.782989 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.782969    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e15c935-b53a-4307-ac3b-66a07729f311-kube-api-access-h22xm" pod "4e15c935-b53a-4307-ac3b-66a07729f311" (UID: "4e15c935-b53a-4307-ac3b-66a07729f311"). InnerVolumeSpecName "kube-api-access-h22xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:08.883289 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.883205    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h22xm\" (UniqueName: \"kubernetes.io/projected/4e15c935-b53a-4307-ac3b-66a07729f311-kube-api-access-h22xm\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:08.883555 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:08.883515    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e15c935-b53a-4307-ac3b-66a07729f311-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:09.553640 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:09.553604    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn"
Nov 04 07:41:09.554145 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:09.553634    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e59041f-cc5hn" event={"ID":"4e15c935-b53a-4307-ac3b-66a07729f311","Type":"ContainerDied","Data":"e57582471c015808c25c71d4b76157ef995da235919b213d698d6acfee3a67ea"}
Nov 04 07:41:09.554145 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:09.553691    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e57582471c015808c25c71d4b76157ef995da235919b213d698d6acfee3a67ea"
Nov 04 07:41:10.096808 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.096774    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" path="/var/lib/kubelet/pods/44b06376-8c4a-45ac-a97e-a6e1cb982b32/volumes"
Nov 04 07:41:10.205948 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.205910    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"]
Nov 04 07:41:10.209557 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.209532    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:10.278498 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.278461    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"]
Nov 04 07:41:10.279159 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.279139    5626 cpu_manager.go:414] "containerMap: removing container" podUID="6749a48e-433c-4073-9d11-1391830d9514" containerName="payload-processing"
Nov 04 07:41:10.279282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.279159    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6749a48e-433c-4073-9d11-1391830d9514" containerName="payload-processing"
Nov 04 07:41:10.279282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.279222    5626 cpu_manager.go:414] "containerMap: removing container" podUID="4e15c935-b53a-4307-ac3b-66a07729f311" containerName="revoke-keys"
Nov 04 07:41:10.279282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.279231    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e15c935-b53a-4307-ac3b-66a07729f311" containerName="revoke-keys"
Nov 04 07:41:10.279282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.279254    5626 cpu_manager.go:414] "containerMap: removing container" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main"
Nov 04 07:41:10.279282 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.279261    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main"
Nov 04 07:41:10.279580 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.279431    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e15c935-b53a-4307-ac3b-66a07729f311" containerName="revoke-keys"
Nov 04 07:41:10.279580 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.279452    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="44b06376-8c4a-45ac-a97e-a6e1cb982b32" containerName="main"
Nov 04 07:41:10.279580 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.279467    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="6749a48e-433c-4073-9d11-1391830d9514" containerName="payload-processing"
Nov 04 07:41:10.283473 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.283450    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.302676 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.302641    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-5cc1fb4bfccd\""
Nov 04 07:41:10.303933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.303882    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"]
Nov 04 07:41:10.310741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.310711    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-provider-credentials\") pod \"payload-processing-external-model-6d5d9997cc-qr9gm\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") " pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.310856 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.310841    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-plugins-config-volume\") pod \"payload-processing-external-model-6d5d9997cc-qr9gm\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") " pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.311140 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.311112    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-routing-overlay-volume\") pod \"payload-processing-external-model-6d5d9997cc-qr9gm\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") " pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.311260 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.311156    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks4ng\" (UniqueName: \"kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-kube-api-access-ks4ng\") pod \"payload-processing-external-model-6d5d9997cc-qr9gm\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") " pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.412273 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.412180    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-routing-overlay-volume\") pod \"payload-processing-external-model-6d5d9997cc-qr9gm\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") " pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.412273 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.412228    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks4ng\" (UniqueName: \"kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-kube-api-access-ks4ng\") pod \"payload-processing-external-model-6d5d9997cc-qr9gm\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") " pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.412273 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.412268    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-provider-credentials\") pod \"payload-processing-external-model-6d5d9997cc-qr9gm\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") " pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.412585 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.412344    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-plugins-config-volume\") pod \"payload-processing-external-model-6d5d9997cc-qr9gm\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") " pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.412955 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.412928    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-routing-overlay-volume\") pod \"payload-processing-external-model-6d5d9997cc-qr9gm\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") " pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.413256 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.413203    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-plugins-config-volume\") pod \"payload-processing-external-model-6d5d9997cc-qr9gm\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") " pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.415249 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.415227    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-provider-credentials\") pod \"payload-processing-external-model-6d5d9997cc-qr9gm\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") " pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.436524 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.436496    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks4ng\" (UniqueName: \"kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-kube-api-access-ks4ng\") pod \"payload-processing-external-model-6d5d9997cc-qr9gm\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") " pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:10.622889 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:10.622579    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:11.058246 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:11.057935    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"]
Nov 04 07:41:11.063292 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:41:11.063257    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14270d6e_139b_487a_8981_e0ac2aac8d0a.slice/crio-b3e39764ef093eaa9881a0ff3b1e6d528ec9af874021336cb9852a6a02066e20 WatchSource:0}: Error finding container b3e39764ef093eaa9881a0ff3b1e6d528ec9af874021336cb9852a6a02066e20: Status 404 returned error can't find the container with id b3e39764ef093eaa9881a0ff3b1e6d528ec9af874021336cb9852a6a02066e20
Nov 04 07:41:11.539337 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:11.539295    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg" podUID="e051d9eb-49d3-4277-8e05-a1ca78585d9f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:41:11.576260 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:11.576227    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm" event={"ID":"14270d6e-139b-487a-8981-e0ac2aac8d0a","Type":"ContainerStarted","Data":"48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f"}
Nov 04 07:41:11.576460 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:11.576293    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm" event={"ID":"14270d6e-139b-487a-8981-e0ac2aac8d0a","Type":"ContainerStarted","Data":"b3e39764ef093eaa9881a0ff3b1e6d528ec9af874021336cb9852a6a02066e20"}
Nov 04 07:41:11.576549 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:11.576483    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:11.598055 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:11.597985    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm" podStartSLOduration=1.59796776 podStartE2EDuration="1.59796776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:41:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:41:11.596650946 +0000 UTC m=+1493.618866207" watchObservedRunningTime="2026-11-04 07:41:11.59796776 +0000 UTC m=+1493.620183025"
Nov 04 07:41:13.373385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:13.371584    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:41:13.464724 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:13.464668    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:41:13.872387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:13.871502    5626 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": context deadline exceeded"
Nov 04 07:41:15.207001 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.206947    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v" podUID="1697d471-a2ca-4151-8a7a-16b50c6d3b2b" containerName="payload-processing" containerID="cri-o://34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9" gracePeriod=25
Nov 04 07:41:15.487941 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.487908    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:15.638821 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.638784    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-provider-credentials\" (UniqueName: \"kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-provider-credentials\") pod \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") "
Nov 04 07:41:15.638990 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.638835    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-routing-overlay-volume\") pod \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") "
Nov 04 07:41:15.638990 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.638927    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-kube-api-access-qch4h\" (UniqueName: \"kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-kube-api-access-qch4h\") pod \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") "
Nov 04 07:41:15.639097 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.639029    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-plugins-config-volume\") pod \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\" (UID: \"1697d471-a2ca-4151-8a7a-16b50c6d3b2b\") "
Nov 04 07:41:15.639325 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.639235    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-routing-overlay-volume" pod "1697d471-a2ca-4151-8a7a-16b50c6d3b2b" (UID: "1697d471-a2ca-4151-8a7a-16b50c6d3b2b"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:15.639497 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.639473    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-plugins-config-volume" pod "1697d471-a2ca-4151-8a7a-16b50c6d3b2b" (UID: "1697d471-a2ca-4151-8a7a-16b50c6d3b2b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:15.639613 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.639597    5626 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-routing-overlay-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:15.639680 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.639616    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:15.641060 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.641032    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-provider-credentials" pod "1697d471-a2ca-4151-8a7a-16b50c6d3b2b" (UID: "1697d471-a2ca-4151-8a7a-16b50c6d3b2b"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:15.641166 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.641122    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-kube-api-access-qch4h" pod "1697d471-a2ca-4151-8a7a-16b50c6d3b2b" (UID: "1697d471-a2ca-4151-8a7a-16b50c6d3b2b"). InnerVolumeSpecName "kube-api-access-qch4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:15.740678 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.740596    5626 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-provider-credentials\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:15.740678 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.740622    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qch4h\" (UniqueName: \"kubernetes.io/projected/1697d471-a2ca-4151-8a7a-16b50c6d3b2b-kube-api-access-qch4h\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:15.784097 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.784062    5626 generic.go:356] "Generic (PLEG): container finished" podID="1697d471-a2ca-4151-8a7a-16b50c6d3b2b" containerID="34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9" exitCode=0
Nov 04 07:41:15.784268 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.784114    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"
Nov 04 07:41:15.784268 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.784148    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v" event={"ID":"1697d471-a2ca-4151-8a7a-16b50c6d3b2b","Type":"ContainerDied","Data":"34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9"}
Nov 04 07:41:15.784268 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.784214    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v" event={"ID":"1697d471-a2ca-4151-8a7a-16b50c6d3b2b","Type":"ContainerDied","Data":"79dac47e0826e9ba2a3652ea8c0f8b072b2ee74d2be47832be1b8ca2c8670928"}
Nov 04 07:41:15.784268 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.784243    5626 scope.go:122] "RemoveContainer" containerID="34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9"
Nov 04 07:41:15.795768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.795749    5626 scope.go:122] "RemoveContainer" containerID="34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9"
Nov 04 07:41:15.796048 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:15.796030    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9\": container with ID starting with 34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9 not found: ID does not exist" containerID="34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9"
Nov 04 07:41:15.796107 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.796054    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9"} err="failed to get container status \"34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9\": rpc error: code = NotFound desc = could not find container \"34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9\": container with ID starting with 34bbc602bc5c197a8fa756569c1767096d2106fe092f25a8846b68ba9c9f3bf9 not found: ID does not exist"
Nov 04 07:41:15.809169 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.809133    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"]
Nov 04 07:41:15.813210 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:15.813187    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v"]
Nov 04 07:41:16.094602 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:16.094573    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1697d471-a2ca-4151-8a7a-16b50c6d3b2b" path="/var/lib/kubelet/pods/1697d471-a2ca-4151-8a7a-16b50c6d3b2b/volumes"
Nov 04 07:41:21.027333 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:21.024837    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"]
Nov 04 07:41:21.048326 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:21.047540    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:21.556524 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:21.556478    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg" podUID="e051d9eb-49d3-4277-8e05-a1ca78585d9f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:41:23.912762 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:23.912726    5626 generic.go:356] "Generic (PLEG): container finished" podID="e051d9eb-49d3-4277-8e05-a1ca78585d9f" containerID="dfac28bec78408d1dc88e407766a855d6b2f7e4c72a4bc8292a85dfd75926e8b" exitCode=0
Nov 04 07:41:23.913156 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:23.912783    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg" event={"ID":"e051d9eb-49d3-4277-8e05-a1ca78585d9f","Type":"ContainerDied","Data":"dfac28bec78408d1dc88e407766a855d6b2f7e4c72a4bc8292a85dfd75926e8b"}
Nov 04 07:41:24.392347 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.392321    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"
Nov 04 07:41:24.459443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.459337    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e051d9eb-49d3-4277-8e05-a1ca78585d9f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e051d9eb-49d3-4277-8e05-a1ca78585d9f-plugins-config-volume\") pod \"e051d9eb-49d3-4277-8e05-a1ca78585d9f\" (UID: \"e051d9eb-49d3-4277-8e05-a1ca78585d9f\") "
Nov 04 07:41:24.459688 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.459519    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e051d9eb-49d3-4277-8e05-a1ca78585d9f-kube-api-access-5xvlc\" (UniqueName: \"kubernetes.io/projected/e051d9eb-49d3-4277-8e05-a1ca78585d9f-kube-api-access-5xvlc\") pod \"e051d9eb-49d3-4277-8e05-a1ca78585d9f\" (UID: \"e051d9eb-49d3-4277-8e05-a1ca78585d9f\") "
Nov 04 07:41:24.460084 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.460055    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e051d9eb-49d3-4277-8e05-a1ca78585d9f-plugins-config-volume" pod "e051d9eb-49d3-4277-8e05-a1ca78585d9f" (UID: "e051d9eb-49d3-4277-8e05-a1ca78585d9f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:24.462868 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.462840    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e051d9eb-49d3-4277-8e05-a1ca78585d9f-kube-api-access-5xvlc" pod "e051d9eb-49d3-4277-8e05-a1ca78585d9f" (UID: "e051d9eb-49d3-4277-8e05-a1ca78585d9f"). InnerVolumeSpecName "kube-api-access-5xvlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:24.480146 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.480116    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"]
Nov 04 07:41:24.480434 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.480399    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy" containerID="cri-o://16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886" gracePeriod=30
Nov 04 07:41:24.561116 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.561082    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xvlc\" (UniqueName: \"kubernetes.io/projected/e051d9eb-49d3-4277-8e05-a1ca78585d9f-kube-api-access-5xvlc\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:24.561116 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.561114    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e051d9eb-49d3-4277-8e05-a1ca78585d9f-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:24.921688 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.921602    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg" event={"ID":"e051d9eb-49d3-4277-8e05-a1ca78585d9f","Type":"ContainerDied","Data":"d376ddf69e790a8e78acec66cc61446620d200b36554d73497111d625be43223"}
Nov 04 07:41:24.921688 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.921658    5626 scope.go:122] "RemoveContainer" containerID="dfac28bec78408d1dc88e407766a855d6b2f7e4c72a4bc8292a85dfd75926e8b"
Nov 04 07:41:24.922178 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.921610    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"
Nov 04 07:41:24.952134 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.952106    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"]
Nov 04 07:41:24.956332 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:24.956307    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg"]
Nov 04 07:41:26.026088 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.026039    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm" podUID="14270d6e-139b-487a-8981-e0ac2aac8d0a" containerName="payload-processing" containerID="cri-o://48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f" gracePeriod=25
Nov 04 07:41:26.093878 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.093849    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e051d9eb-49d3-4277-8e05-a1ca78585d9f" path="/var/lib/kubelet/pods/e051d9eb-49d3-4277-8e05-a1ca78585d9f/volumes"
Nov 04 07:41:26.284050 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.284024    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:26.387296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.387262    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-provider-credentials\" (UniqueName: \"kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-provider-credentials\") pod \"14270d6e-139b-487a-8981-e0ac2aac8d0a\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") "
Nov 04 07:41:26.387495 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.387305    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-plugins-config-volume\") pod \"14270d6e-139b-487a-8981-e0ac2aac8d0a\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") "
Nov 04 07:41:26.387495 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.387450    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-routing-overlay-volume\") pod \"14270d6e-139b-487a-8981-e0ac2aac8d0a\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") "
Nov 04 07:41:26.387609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.387588    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-kube-api-access-ks4ng\" (UniqueName: \"kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-kube-api-access-ks4ng\") pod \"14270d6e-139b-487a-8981-e0ac2aac8d0a\" (UID: \"14270d6e-139b-487a-8981-e0ac2aac8d0a\") "
Nov 04 07:41:26.387802 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.387777    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-plugins-config-volume" pod "14270d6e-139b-487a-8981-e0ac2aac8d0a" (UID: "14270d6e-139b-487a-8981-e0ac2aac8d0a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:26.387878 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.387818    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-routing-overlay-volume" pod "14270d6e-139b-487a-8981-e0ac2aac8d0a" (UID: "14270d6e-139b-487a-8981-e0ac2aac8d0a"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:26.388187 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.388165    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:26.388187 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.388187    5626 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/14270d6e-139b-487a-8981-e0ac2aac8d0a-routing-overlay-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:26.389773 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.389748    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-provider-credentials" pod "14270d6e-139b-487a-8981-e0ac2aac8d0a" (UID: "14270d6e-139b-487a-8981-e0ac2aac8d0a"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:26.389865 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.389817    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-kube-api-access-ks4ng" pod "14270d6e-139b-487a-8981-e0ac2aac8d0a" (UID: "14270d6e-139b-487a-8981-e0ac2aac8d0a"). InnerVolumeSpecName "kube-api-access-ks4ng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:26.490036 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.489986    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ks4ng\" (UniqueName: \"kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-kube-api-access-ks4ng\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:26.490036 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.490018    5626 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/14270d6e-139b-487a-8981-e0ac2aac8d0a-provider-credentials\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:26.531664 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.531635    5626 patch_prober.go:29] interesting pod/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 07:41:26.531804 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.531709    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 07:41:26.938558 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.938524    5626 generic.go:356] "Generic (PLEG): container finished" podID="14270d6e-139b-487a-8981-e0ac2aac8d0a" containerID="48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f" exitCode=0
Nov 04 07:41:26.938742 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.938575    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"
Nov 04 07:41:26.938742 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.938611    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm" event={"ID":"14270d6e-139b-487a-8981-e0ac2aac8d0a","Type":"ContainerDied","Data":"48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f"}
Nov 04 07:41:26.938742 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.938673    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm" event={"ID":"14270d6e-139b-487a-8981-e0ac2aac8d0a","Type":"ContainerDied","Data":"b3e39764ef093eaa9881a0ff3b1e6d528ec9af874021336cb9852a6a02066e20"}
Nov 04 07:41:26.938742 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.938697    5626 scope.go:122] "RemoveContainer" containerID="48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f"
Nov 04 07:41:26.950174 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.950157    5626 scope.go:122] "RemoveContainer" containerID="48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f"
Nov 04 07:41:26.950498 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:26.950478    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f\": container with ID starting with 48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f not found: ID does not exist" containerID="48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f"
Nov 04 07:41:26.950561 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.950508    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f"} err="failed to get container status \"48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f\": rpc error: code = NotFound desc = could not find container \"48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f\": container with ID starting with 48f50bf240d3a8ee95d513ab28aa56bca5793ce63ec67155877af3dd889d3a1f not found: ID does not exist"
Nov 04 07:41:26.963670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.963647    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"]
Nov 04 07:41:26.970823 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:26.970797    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm"]
Nov 04 07:41:27.605796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:27.605760    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"]
Nov 04 07:41:28.115877 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:28.115826    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14270d6e-139b-487a-8981-e0ac2aac8d0a" path="/var/lib/kubelet/pods/14270d6e-139b-487a-8981-e0ac2aac8d0a/volumes"
Nov 04 07:41:29.766253 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.766225    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:41:29.843062 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.843025    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db38048a-323a-4ab4-a60a-a394546f78f6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db38048a-323a-4ab4-a60a-a394546f78f6-istiod-ca-cert\") pod \"db38048a-323a-4ab4-a60a-a394546f78f6\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") "
Nov 04 07:41:29.843273 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.843121    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-kube-api-access-ftqwj\" (UniqueName: \"kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-kube-api-access-ftqwj\") pod \"db38048a-323a-4ab4-a60a-a394546f78f6\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") "
Nov 04 07:41:29.843273 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.843219    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-data\") pod \"db38048a-323a-4ab4-a60a-a394546f78f6\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") "
Nov 04 07:41:29.843273 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.843268    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/db38048a-323a-4ab4-a60a-a394546f78f6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db38048a-323a-4ab4-a60a-a394546f78f6-istio-podinfo\") pod \"db38048a-323a-4ab4-a60a-a394546f78f6\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") "
Nov 04 07:41:29.843500 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.843299    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-istio-token\" (UniqueName: \"kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-istio-token\") pod \"db38048a-323a-4ab4-a60a-a394546f78f6\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") "
Nov 04 07:41:29.843500 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.843411    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-envoy\") pod \"db38048a-323a-4ab4-a60a-a394546f78f6\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") "
Nov 04 07:41:29.843500 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.843447    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-certs\") pod \"db38048a-323a-4ab4-a60a-a394546f78f6\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") "
Nov 04 07:41:29.843500 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.843486    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-socket\") pod \"db38048a-323a-4ab4-a60a-a394546f78f6\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") "
Nov 04 07:41:29.843717 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.843624    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-credential-socket\") pod \"db38048a-323a-4ab4-a60a-a394546f78f6\" (UID: \"db38048a-323a-4ab4-a60a-a394546f78f6\") "
Nov 04 07:41:29.848674 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.844595    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-credential-socket" pod "db38048a-323a-4ab4-a60a-a394546f78f6" (UID: "db38048a-323a-4ab4-a60a-a394546f78f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:29.848674 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.845906    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-socket" pod "db38048a-323a-4ab4-a60a-a394546f78f6" (UID: "db38048a-323a-4ab4-a60a-a394546f78f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:29.848674 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.846117    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-certs" pod "db38048a-323a-4ab4-a60a-a394546f78f6" (UID: "db38048a-323a-4ab4-a60a-a394546f78f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:29.848674 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.846168    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-data" pod "db38048a-323a-4ab4-a60a-a394546f78f6" (UID: "db38048a-323a-4ab4-a60a-a394546f78f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:29.848674 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.846391    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db38048a-323a-4ab4-a60a-a394546f78f6-istiod-ca-cert" pod "db38048a-323a-4ab4-a60a-a394546f78f6" (UID: "db38048a-323a-4ab4-a60a-a394546f78f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:29.851150 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.851085    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db38048a-323a-4ab4-a60a-a394546f78f6-istio-podinfo" pod "db38048a-323a-4ab4-a60a-a394546f78f6" (UID: "db38048a-323a-4ab4-a60a-a394546f78f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:41:29.852687 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.851640    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-istio-token" pod "db38048a-323a-4ab4-a60a-a394546f78f6" (UID: "db38048a-323a-4ab4-a60a-a394546f78f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:29.856518 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.856483    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-kube-api-access-ftqwj" pod "db38048a-323a-4ab4-a60a-a394546f78f6" (UID: "db38048a-323a-4ab4-a60a-a394546f78f6"). InnerVolumeSpecName "kube-api-access-ftqwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:29.862454 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.862409    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-envoy" pod "db38048a-323a-4ab4-a60a-a394546f78f6" (UID: "db38048a-323a-4ab4-a60a-a394546f78f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:29.945334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.945300    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:29.945334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.945330    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:29.945334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.945342    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:29.945617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.945353    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:29.945617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.945402    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db38048a-323a-4ab4-a60a-a394546f78f6-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:29.945617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.945412    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftqwj\" (UniqueName: \"kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-kube-api-access-ftqwj\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:29.945617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.945421    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db38048a-323a-4ab4-a60a-a394546f78f6-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:29.945617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.945430    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db38048a-323a-4ab4-a60a-a394546f78f6-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:29.945617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:29.945439    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db38048a-323a-4ab4-a60a-a394546f78f6-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:30.005112 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.005029    5626 generic.go:356] "Generic (PLEG): container finished" podID="db38048a-323a-4ab4-a60a-a394546f78f6" containerID="16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886" exitCode=0
Nov 04 07:41:30.005112 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.005086    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"
Nov 04 07:41:30.005302 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.005108    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" event={"ID":"db38048a-323a-4ab4-a60a-a394546f78f6","Type":"ContainerDied","Data":"16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886"}
Nov 04 07:41:30.005302 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.005170    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp" event={"ID":"db38048a-323a-4ab4-a60a-a394546f78f6","Type":"ContainerDied","Data":"4b0f006cb725801ebf68f02f189619c772dfff876453f36d8d13c82410b80f8a"}
Nov 04 07:41:30.005302 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.005200    5626 scope.go:122] "RemoveContainer" containerID="16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886"
Nov 04 07:41:30.016455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.016434    5626 scope.go:122] "RemoveContainer" containerID="bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b"
Nov 04 07:41:30.026602 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.026583    5626 scope.go:122] "RemoveContainer" containerID="16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886"
Nov 04 07:41:30.026880 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:30.026862    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886\": container with ID starting with 16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886 not found: ID does not exist" containerID="16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886"
Nov 04 07:41:30.026917 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.026890    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886"} err="failed to get container status \"16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886\": rpc error: code = NotFound desc = could not find container \"16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886\": container with ID starting with 16a189e0d86c59c88b0665f7e813d427e9e60dc716c2b4aa434b8a61b61e6886 not found: ID does not exist"
Nov 04 07:41:30.026917 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.026906    5626 scope.go:122] "RemoveContainer" containerID="bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b"
Nov 04 07:41:30.027134 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:30.027119    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b\": container with ID starting with bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b not found: ID does not exist" containerID="bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b"
Nov 04 07:41:30.027181 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.027138    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b"} err="failed to get container status \"bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b\": rpc error: code = NotFound desc = could not find container \"bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b\": container with ID starting with bb23e0d6f5bea2a6989f335850cbbaa621d521f30f40db2438f207f9410ba92b not found: ID does not exist"
Nov 04 07:41:30.033934 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.033914    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"]
Nov 04 07:41:30.039438 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.039417    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp"]
Nov 04 07:41:30.093055 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:30.093031    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" path="/var/lib/kubelet/pods/db38048a-323a-4ab4-a60a-a394546f78f6/volumes"
Nov 04 07:41:32.568274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:32.568222    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"]
Nov 04 07:41:32.569602 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:32.569567    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerName="istio-proxy" containerID="cri-o://ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f" gracePeriod=30
Nov 04 07:41:32.612531 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:32.612456    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6" podUID="980bbac5-cf4e-4bbd-9fd6-60bf29548e37" containerName="payload-pre-processing" containerID="cri-o://0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b" gracePeriod=25
Nov 04 07:41:32.624033 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:32.624003    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"]
Nov 04 07:41:32.648888 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:32.648854    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"]
Nov 04 07:41:32.735918 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:32.735128    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6" podUID="980bbac5-cf4e-4bbd-9fd6-60bf29548e37" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:41:33.977812 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:33.977774    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"]
Nov 04 07:41:35.033643 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:35.033602    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-788ls container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:41:35.034315 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:35.034282    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls" podUID="5be66e14-e024-4afe-8e07-9cd3cc138ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:41:35.070919 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:35.070870    5626 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-g5rgr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:41:35.071203 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:35.071167    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr" podUID="614988be-64cf-4a39-b201-ff69a7f7fb19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:41:36.172996 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:36.172961    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:41:36.562481 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.562445    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"]
Nov 04 07:41:36.563417 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563394    5626 cpu_manager.go:414] "containerMap: removing container" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy"
Nov 04 07:41:36.563487 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563420    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy"
Nov 04 07:41:36.563487 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563442    5626 cpu_manager.go:414] "containerMap: removing container" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy"
Nov 04 07:41:36.563487 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563452    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy"
Nov 04 07:41:36.563590 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563487    5626 cpu_manager.go:414] "containerMap: removing container" podUID="1697d471-a2ca-4151-8a7a-16b50c6d3b2b" containerName="payload-processing"
Nov 04 07:41:36.563590 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563495    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1697d471-a2ca-4151-8a7a-16b50c6d3b2b" containerName="payload-processing"
Nov 04 07:41:36.563590 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563515    5626 cpu_manager.go:414] "containerMap: removing container" podUID="14270d6e-139b-487a-8981-e0ac2aac8d0a" containerName="payload-processing"
Nov 04 07:41:36.563590 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563522    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14270d6e-139b-487a-8981-e0ac2aac8d0a" containerName="payload-processing"
Nov 04 07:41:36.563590 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563539    5626 cpu_manager.go:414] "containerMap: removing container" podUID="e051d9eb-49d3-4277-8e05-a1ca78585d9f" containerName="payload-pre-processing"
Nov 04 07:41:36.563590 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563547    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e051d9eb-49d3-4277-8e05-a1ca78585d9f" containerName="payload-pre-processing"
Nov 04 07:41:36.563769 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563729    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="1697d471-a2ca-4151-8a7a-16b50c6d3b2b" containerName="payload-processing"
Nov 04 07:41:36.563769 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563744    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="14270d6e-139b-487a-8981-e0ac2aac8d0a" containerName="payload-processing"
Nov 04 07:41:36.563769 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563757    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="e051d9eb-49d3-4277-8e05-a1ca78585d9f" containerName="payload-pre-processing"
Nov 04 07:41:36.563868 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563775    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy"
Nov 04 07:41:36.563868 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.563787    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="db38048a-323a-4ab4-a60a-a394546f78f6" containerName="istio-proxy"
Nov 04 07:41:36.574340 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.574302    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"]
Nov 04 07:41:36.574525 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.574458    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:36.578426 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.578394    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:41:36.578558 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.578462    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:41:36.581289 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.581267    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:41:36.582912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.582656    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:41:36.586011 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.585983    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gjs6\" (UniqueName: \"kubernetes.io/projected/b9798d72-17cb-421b-836d-942de901c436-kube-api-access-9gjs6\") pod \"maas-api-revoke-sub-c039095a-thj7r\" (UID: \"b9798d72-17cb-421b-836d-942de901c436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:36.586178 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.586157    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b9798d72-17cb-421b-836d-942de901c436-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c039095a-thj7r\" (UID: \"b9798d72-17cb-421b-836d-942de901c436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:36.586274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.586216    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b9798d72-17cb-421b-836d-942de901c436-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c039095a-thj7r\" (UID: \"b9798d72-17cb-421b-836d-942de901c436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:36.687167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.687137    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b9798d72-17cb-421b-836d-942de901c436-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c039095a-thj7r\" (UID: \"b9798d72-17cb-421b-836d-942de901c436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:36.687415 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.687191    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b9798d72-17cb-421b-836d-942de901c436-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c039095a-thj7r\" (UID: \"b9798d72-17cb-421b-836d-942de901c436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:36.687415 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.687311    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gjs6\" (UniqueName: \"kubernetes.io/projected/b9798d72-17cb-421b-836d-942de901c436-kube-api-access-9gjs6\") pod \"maas-api-revoke-sub-c039095a-thj7r\" (UID: \"b9798d72-17cb-421b-836d-942de901c436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:36.687898 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.687871    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b9798d72-17cb-421b-836d-942de901c436-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c039095a-thj7r\" (UID: \"b9798d72-17cb-421b-836d-942de901c436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:36.689641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.689621    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b9798d72-17cb-421b-836d-942de901c436-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c039095a-thj7r\" (UID: \"b9798d72-17cb-421b-836d-942de901c436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:36.699441 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.699419    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gjs6\" (UniqueName: \"kubernetes.io/projected/b9798d72-17cb-421b-836d-942de901c436-kube-api-access-9gjs6\") pod \"maas-api-revoke-sub-c039095a-thj7r\" (UID: \"b9798d72-17cb-421b-836d-942de901c436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:36.906435 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:36.905520    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:37.215880 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:37.215842    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"]
Nov 04 07:41:37.217115 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:41:37.217088    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9798d72_17cb_421b_836d_942de901c436.slice/crio-0095dcb474edf7b6f05637a3c2b1fa7e984e0f5260e8a197b781113fa4a0d8b3 WatchSource:0}: Error finding container 0095dcb474edf7b6f05637a3c2b1fa7e984e0f5260e8a197b781113fa4a0d8b3: Status 404 returned error can't find the container with id 0095dcb474edf7b6f05637a3c2b1fa7e984e0f5260e8a197b781113fa4a0d8b3
Nov 04 07:41:37.240015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:37.239991    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r" event={"ID":"b9798d72-17cb-421b-836d-942de901c436","Type":"ContainerStarted","Data":"0095dcb474edf7b6f05637a3c2b1fa7e984e0f5260e8a197b781113fa4a0d8b3"}
Nov 04 07:41:37.624709 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:37.624653    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg" podUID="0f351198-a3ef-4598-a748-26b51b757a0a" containerName="payload-processing" containerID="cri-o://a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1" gracePeriod=25
Nov 04 07:41:37.649263 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:37.649225    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t" podUID="2702960b-b62a-4bfd-8457-13eb82b88043" containerName="payload-pre-processing" containerID="cri-o://aabdfcd4c12ce7ce6d3343867c9583795c4f69d1ea652a5cbd376bc9fcf4f84b" gracePeriod=25
Nov 04 07:41:37.951868 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:37.951830    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:41:37.999916 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:37.999887    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"
Nov 04 07:41:38.016095 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016060    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-credential-socket\") pod \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") "
Nov 04 07:41:38.016274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016124    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-certs\") pod \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") "
Nov 04 07:41:38.016274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016197    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istiod-ca-cert\") pod \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") "
Nov 04 07:41:38.016274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016266    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f351198-a3ef-4598-a748-26b51b757a0a-kube-api-access-bs4lh\" (UniqueName: \"kubernetes.io/projected/0f351198-a3ef-4598-a748-26b51b757a0a-kube-api-access-bs4lh\") pod \"0f351198-a3ef-4598-a748-26b51b757a0a\" (UID: \"0f351198-a3ef-4598-a748-26b51b757a0a\") "
Nov 04 07:41:38.016513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016306    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-kube-api-access-d2xzs\" (UniqueName: \"kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-kube-api-access-d2xzs\") pod \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") "
Nov 04 07:41:38.016513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016331    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-credential-socket" pod "693fd0ce-1ebc-4173-9d22-b8ed367c1a26" (UID: "693fd0ce-1ebc-4173-9d22-b8ed367c1a26"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:38.016513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016393    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-socket\") pod \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") "
Nov 04 07:41:38.016513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016491    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-podinfo\") pod \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") "
Nov 04 07:41:38.016728 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016515    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-data\") pod \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") "
Nov 04 07:41:38.016728 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016623    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f351198-a3ef-4598-a748-26b51b757a0a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f351198-a3ef-4598-a748-26b51b757a0a-plugins-config-volume\") pod \"0f351198-a3ef-4598-a748-26b51b757a0a\" (UID: \"0f351198-a3ef-4598-a748-26b51b757a0a\") "
Nov 04 07:41:38.016728 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016674    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-envoy\") pod \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") "
Nov 04 07:41:38.016728 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016703    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-token\" (UniqueName: \"kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-token\") pod \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\" (UID: \"693fd0ce-1ebc-4173-9d22-b8ed367c1a26\") "
Nov 04 07:41:38.016944 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016725    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istiod-ca-cert" pod "693fd0ce-1ebc-4173-9d22-b8ed367c1a26" (UID: "693fd0ce-1ebc-4173-9d22-b8ed367c1a26"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:38.017007 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.016967    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-certs" pod "693fd0ce-1ebc-4173-9d22-b8ed367c1a26" (UID: "693fd0ce-1ebc-4173-9d22-b8ed367c1a26"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:38.017299 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.017271    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:38.017451 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.017306    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:38.017451 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.017321    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:38.018055 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.017615    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-socket" pod "693fd0ce-1ebc-4173-9d22-b8ed367c1a26" (UID: "693fd0ce-1ebc-4173-9d22-b8ed367c1a26"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:38.018055 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.017806    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-data" pod "693fd0ce-1ebc-4173-9d22-b8ed367c1a26" (UID: "693fd0ce-1ebc-4173-9d22-b8ed367c1a26"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:38.019511 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.019486    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f351198-a3ef-4598-a748-26b51b757a0a-plugins-config-volume" pod "0f351198-a3ef-4598-a748-26b51b757a0a" (UID: "0f351198-a3ef-4598-a748-26b51b757a0a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:38.026685 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.026636    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-podinfo" pod "693fd0ce-1ebc-4173-9d22-b8ed367c1a26" (UID: "693fd0ce-1ebc-4173-9d22-b8ed367c1a26"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:41:38.026685 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.026646    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-envoy" pod "693fd0ce-1ebc-4173-9d22-b8ed367c1a26" (UID: "693fd0ce-1ebc-4173-9d22-b8ed367c1a26"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:38.026858 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.026713    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-token" pod "693fd0ce-1ebc-4173-9d22-b8ed367c1a26" (UID: "693fd0ce-1ebc-4173-9d22-b8ed367c1a26"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:38.026858 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.026759    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-kube-api-access-d2xzs" pod "693fd0ce-1ebc-4173-9d22-b8ed367c1a26" (UID: "693fd0ce-1ebc-4173-9d22-b8ed367c1a26"). InnerVolumeSpecName "kube-api-access-d2xzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:38.029625 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.029589    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f351198-a3ef-4598-a748-26b51b757a0a-kube-api-access-bs4lh" pod "0f351198-a3ef-4598-a748-26b51b757a0a" (UID: "0f351198-a3ef-4598-a748-26b51b757a0a"). InnerVolumeSpecName "kube-api-access-bs4lh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:38.118664 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.118636    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs4lh\" (UniqueName: \"kubernetes.io/projected/0f351198-a3ef-4598-a748-26b51b757a0a-kube-api-access-bs4lh\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:38.118796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.118665    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2xzs\" (UniqueName: \"kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-kube-api-access-d2xzs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:38.118796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.118681    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:38.118796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.118694    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:38.118796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.118708    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:38.118796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.118720    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f351198-a3ef-4598-a748-26b51b757a0a-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:38.118796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.118735    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:38.118796 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.118748    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693fd0ce-1ebc-4173-9d22-b8ed367c1a26-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:38.251177 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.251147    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r" event={"ID":"b9798d72-17cb-421b-836d-942de901c436","Type":"ContainerStarted","Data":"c4508b087bb699f6465ad2c50743d288d6975a091b235a1e3f7a0586458ac0d8"}
Nov 04 07:41:38.252528 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.252502    5626 generic.go:356] "Generic (PLEG): container finished" podID="0f351198-a3ef-4598-a748-26b51b757a0a" containerID="a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1" exitCode=0
Nov 04 07:41:38.252659 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.252540    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"
Nov 04 07:41:38.252659 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.252588    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg" event={"ID":"0f351198-a3ef-4598-a748-26b51b757a0a","Type":"ContainerDied","Data":"a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1"}
Nov 04 07:41:38.252659 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.252637    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg" event={"ID":"0f351198-a3ef-4598-a748-26b51b757a0a","Type":"ContainerDied","Data":"e07e81032e519fe1ed5d5ea73d370b54cacce22d212a594e5961e33f42bd5e55"}
Nov 04 07:41:38.252805 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.252666    5626 scope.go:122] "RemoveContainer" containerID="a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1"
Nov 04 07:41:38.254586 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.254564    5626 generic.go:356] "Generic (PLEG): container finished" podID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerID="ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f" exitCode=0
Nov 04 07:41:38.254710 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.254615    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"
Nov 04 07:41:38.254710 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.254622    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" event={"ID":"693fd0ce-1ebc-4173-9d22-b8ed367c1a26","Type":"ContainerDied","Data":"ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f"}
Nov 04 07:41:38.254710 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.254652    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x" event={"ID":"693fd0ce-1ebc-4173-9d22-b8ed367c1a26","Type":"ContainerDied","Data":"444680534a88e2e43c3d330b903392348bde26a1ab1e6ef4bdb80358bcf2d75a"}
Nov 04 07:41:38.293812 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.293790    5626 scope.go:122] "RemoveContainer" containerID="a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1"
Nov 04 07:41:38.294128 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:38.294108    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1\": container with ID starting with a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1 not found: ID does not exist" containerID="a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1"
Nov 04 07:41:38.294180 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.294138    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1"} err="failed to get container status \"a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1\": rpc error: code = NotFound desc = could not find container \"a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1\": container with ID starting with a5ec0bb5796867511d2712c99f302e35321eb212e6323d24fef0bdc2b5f92bf1 not found: ID does not exist"
Nov 04 07:41:38.294180 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.294155    5626 scope.go:122] "RemoveContainer" containerID="ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f"
Nov 04 07:41:38.304395 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.304349    5626 scope.go:122] "RemoveContainer" containerID="098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2"
Nov 04 07:41:38.315019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.314993    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"]
Nov 04 07:41:38.317533 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.317506    5626 scope.go:122] "RemoveContainer" containerID="ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f"
Nov 04 07:41:38.317970 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:38.317939    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f\": container with ID starting with ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f not found: ID does not exist" containerID="ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f"
Nov 04 07:41:38.318065 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.317978    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f"} err="failed to get container status \"ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f\": rpc error: code = NotFound desc = could not find container \"ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f\": container with ID starting with ab11cb0c8f453225fc7c3dd28e11b07c26b0493723cc60eb39225bbf20843e6f not found: ID does not exist"
Nov 04 07:41:38.318065 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.318005    5626 scope.go:122] "RemoveContainer" containerID="098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2"
Nov 04 07:41:38.318349 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:38.318333    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2\": container with ID starting with 098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2 not found: ID does not exist" containerID="098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2"
Nov 04 07:41:38.318461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.318353    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2"} err="failed to get container status \"098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2\": rpc error: code = NotFound desc = could not find container \"098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2\": container with ID starting with 098a25e04079e25c8359e5e54a1091d5ea413782bd2e365f675137187e327ec2 not found: ID does not exist"
Nov 04 07:41:38.319827 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.319807    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x"]
Nov 04 07:41:38.332271 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.332245    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"]
Nov 04 07:41:38.335320 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.335295    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg"]
Nov 04 07:41:38.985567 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:38.985480    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4" podUID="0f930022-3f89-466e-bbd9-f545fdc8353c" containerName="payload-processing" containerID="cri-o://1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376" gracePeriod=25
Nov 04 07:41:39.258085 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.258058    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:41:39.269386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.269346    5626 generic.go:356] "Generic (PLEG): container finished" podID="0f930022-3f89-466e-bbd9-f545fdc8353c" containerID="1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376" exitCode=0
Nov 04 07:41:39.269550 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.269426    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"
Nov 04 07:41:39.269550 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.269432    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4" event={"ID":"0f930022-3f89-466e-bbd9-f545fdc8353c","Type":"ContainerDied","Data":"1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376"}
Nov 04 07:41:39.269550 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.269486    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4" event={"ID":"0f930022-3f89-466e-bbd9-f545fdc8353c","Type":"ContainerDied","Data":"71e6194609ffee45fa33d99668e60454facbbbc57b72b4d34a2bb9be7a4ae273"}
Nov 04 07:41:39.269550 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.269513    5626 scope.go:122] "RemoveContainer" containerID="1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376"
Nov 04 07:41:39.270955 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.270933    5626 generic.go:356] "Generic (PLEG): container finished" podID="b9798d72-17cb-421b-836d-942de901c436" containerID="c4508b087bb699f6465ad2c50743d288d6975a091b235a1e3f7a0586458ac0d8" exitCode=0
Nov 04 07:41:39.271058 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.271009    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r" event={"ID":"b9798d72-17cb-421b-836d-942de901c436","Type":"ContainerDied","Data":"c4508b087bb699f6465ad2c50743d288d6975a091b235a1e3f7a0586458ac0d8"}
Nov 04 07:41:39.279800 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.279780    5626 scope.go:122] "RemoveContainer" containerID="1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376"
Nov 04 07:41:39.280088 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:39.280071    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376\": container with ID starting with 1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376 not found: ID does not exist" containerID="1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376"
Nov 04 07:41:39.280139 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.280097    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376"} err="failed to get container status \"1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376\": rpc error: code = NotFound desc = could not find container \"1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376\": container with ID starting with 1bc0d506a5ffe07a8fa5c43d01e4c335e6688ab1c76f3c3572dc81eaeaae5376 not found: ID does not exist"
Nov 04 07:41:39.332464 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.332426    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-plugins-config-volume\") pod \"0f930022-3f89-466e-bbd9-f545fdc8353c\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") "
Nov 04 07:41:39.332649 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.332593    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-provider-credentials\" (UniqueName: \"kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-provider-credentials\") pod \"0f930022-3f89-466e-bbd9-f545fdc8353c\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") "
Nov 04 07:41:39.332918 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.332878    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-plugins-config-volume" pod "0f930022-3f89-466e-bbd9-f545fdc8353c" (UID: "0f930022-3f89-466e-bbd9-f545fdc8353c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:39.333059 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.333037    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-routing-overlay-volume\") pod \"0f930022-3f89-466e-bbd9-f545fdc8353c\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") "
Nov 04 07:41:39.333130 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.333116    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-kube-api-access-nrtx6\" (UniqueName: \"kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-kube-api-access-nrtx6\") pod \"0f930022-3f89-466e-bbd9-f545fdc8353c\" (UID: \"0f930022-3f89-466e-bbd9-f545fdc8353c\") "
Nov 04 07:41:39.333956 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.333932    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:39.334543 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.334520    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-routing-overlay-volume" pod "0f930022-3f89-466e-bbd9-f545fdc8353c" (UID: "0f930022-3f89-466e-bbd9-f545fdc8353c"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:39.337228 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.337150    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-kube-api-access-nrtx6" pod "0f930022-3f89-466e-bbd9-f545fdc8353c" (UID: "0f930022-3f89-466e-bbd9-f545fdc8353c"). InnerVolumeSpecName "kube-api-access-nrtx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:39.359996 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.359943    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-provider-credentials" pod "0f930022-3f89-466e-bbd9-f545fdc8353c" (UID: "0f930022-3f89-466e-bbd9-f545fdc8353c"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:39.434549 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.434524    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:39.434810 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.434792    5626 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-provider-credentials\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:39.434865 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.434816    5626 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0f930022-3f89-466e-bbd9-f545fdc8353c-routing-overlay-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:39.434865 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.434831    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrtx6\" (UniqueName: \"kubernetes.io/projected/0f930022-3f89-466e-bbd9-f545fdc8353c-kube-api-access-nrtx6\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:39.535631 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.535600    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9798d72-17cb-421b-836d-942de901c436-kube-api-access-9gjs6\" (UniqueName: \"kubernetes.io/projected/b9798d72-17cb-421b-836d-942de901c436-kube-api-access-9gjs6\") pod \"b9798d72-17cb-421b-836d-942de901c436\" (UID: \"b9798d72-17cb-421b-836d-942de901c436\") "
Nov 04 07:41:39.535784 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.535724    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9798d72-17cb-421b-836d-942de901c436-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b9798d72-17cb-421b-836d-942de901c436-maas-api-openshift-ca\") pod \"b9798d72-17cb-421b-836d-942de901c436\" (UID: \"b9798d72-17cb-421b-836d-942de901c436\") "
Nov 04 07:41:39.535827 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.535794    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9798d72-17cb-421b-836d-942de901c436-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b9798d72-17cb-421b-836d-942de901c436-maas-api-xks-ca\") pod \"b9798d72-17cb-421b-836d-942de901c436\" (UID: \"b9798d72-17cb-421b-836d-942de901c436\") "
Nov 04 07:41:39.536147 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.536117    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9798d72-17cb-421b-836d-942de901c436-maas-api-openshift-ca" pod "b9798d72-17cb-421b-836d-942de901c436" (UID: "b9798d72-17cb-421b-836d-942de901c436"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:39.536264 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.536242    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b9798d72-17cb-421b-836d-942de901c436-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:39.537666 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.537647    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9798d72-17cb-421b-836d-942de901c436-kube-api-access-9gjs6" pod "b9798d72-17cb-421b-836d-942de901c436" (UID: "b9798d72-17cb-421b-836d-942de901c436"). InnerVolumeSpecName "kube-api-access-9gjs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:39.537769 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.537747    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9798d72-17cb-421b-836d-942de901c436-maas-api-xks-ca" pod "b9798d72-17cb-421b-836d-942de901c436" (UID: "b9798d72-17cb-421b-836d-942de901c436"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:41:39.594164 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.594136    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"]
Nov 04 07:41:39.598475 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.598449    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4"]
Nov 04 07:41:39.636897 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.636868    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b9798d72-17cb-421b-836d-942de901c436-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:39.636897 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:39.636893    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gjs6\" (UniqueName: \"kubernetes.io/projected/b9798d72-17cb-421b-836d-942de901c436-kube-api-access-9gjs6\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:40.100920 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:40.100881    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f351198-a3ef-4598-a748-26b51b757a0a" path="/var/lib/kubelet/pods/0f351198-a3ef-4598-a748-26b51b757a0a/volumes"
Nov 04 07:41:40.101482 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:40.101468    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f930022-3f89-466e-bbd9-f545fdc8353c" path="/var/lib/kubelet/pods/0f930022-3f89-466e-bbd9-f545fdc8353c/volumes"
Nov 04 07:41:40.102027 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:40.102014    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" path="/var/lib/kubelet/pods/693fd0ce-1ebc-4173-9d22-b8ed367c1a26/volumes"
Nov 04 07:41:40.282816 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:40.282790    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r"
Nov 04 07:41:40.283226 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:40.282787    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c039095a-thj7r" event={"ID":"b9798d72-17cb-421b-836d-942de901c436","Type":"ContainerDied","Data":"0095dcb474edf7b6f05637a3c2b1fa7e984e0f5260e8a197b781113fa4a0d8b3"}
Nov 04 07:41:40.283226 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:40.282910    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0095dcb474edf7b6f05637a3c2b1fa7e984e0f5260e8a197b781113fa4a0d8b3"
Nov 04 07:41:41.653701 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.653665    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"]
Nov 04 07:41:41.654449 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654426    5626 cpu_manager.go:414] "containerMap: removing container" podUID="0f930022-3f89-466e-bbd9-f545fdc8353c" containerName="payload-processing"
Nov 04 07:41:41.654449 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654448    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f930022-3f89-466e-bbd9-f545fdc8353c" containerName="payload-processing"
Nov 04 07:41:41.654629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654462    5626 cpu_manager.go:414] "containerMap: removing container" podUID="b9798d72-17cb-421b-836d-942de901c436" containerName="revoke-keys"
Nov 04 07:41:41.654629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654472    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9798d72-17cb-421b-836d-942de901c436" containerName="revoke-keys"
Nov 04 07:41:41.654629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654494    5626 cpu_manager.go:414] "containerMap: removing container" podUID="0f351198-a3ef-4598-a748-26b51b757a0a" containerName="payload-processing"
Nov 04 07:41:41.654629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654502    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f351198-a3ef-4598-a748-26b51b757a0a" containerName="payload-processing"
Nov 04 07:41:41.654629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654525    5626 cpu_manager.go:414] "containerMap: removing container" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerName="istio-proxy"
Nov 04 07:41:41.654629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654532    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerName="istio-proxy"
Nov 04 07:41:41.654629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654549    5626 cpu_manager.go:414] "containerMap: removing container" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerName="istio-proxy"
Nov 04 07:41:41.654629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654555    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerName="istio-proxy"
Nov 04 07:41:41.655003 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654711    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerName="istio-proxy"
Nov 04 07:41:41.655003 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654727    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="693fd0ce-1ebc-4173-9d22-b8ed367c1a26" containerName="istio-proxy"
Nov 04 07:41:41.655003 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654745    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f930022-3f89-466e-bbd9-f545fdc8353c" containerName="payload-processing"
Nov 04 07:41:41.655003 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654754    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9798d72-17cb-421b-836d-942de901c436" containerName="revoke-keys"
Nov 04 07:41:41.655003 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.654766    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f351198-a3ef-4598-a748-26b51b757a0a" containerName="payload-processing"
Nov 04 07:41:41.659698 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.659677    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:41.664174 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.664149    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:41:41.664313 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.664268    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:41:41.664587 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.664564    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:41:41.666123 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.666099    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:41:41.667592 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.667572    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"]
Nov 04 07:41:41.783385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.783337    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c3fcc35a-jjmf2\" (UID: \"216d4709-2b24-4351-bea6-b2d686dab302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:41.783662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.783634    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c3fcc35a-jjmf2\" (UID: \"216d4709-2b24-4351-bea6-b2d686dab302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:41.783721 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.783685    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nm5x\" (UniqueName: \"kubernetes.io/projected/216d4709-2b24-4351-bea6-b2d686dab302-kube-api-access-2nm5x\") pod \"maas-api-revoke-sub-c3fcc35a-jjmf2\" (UID: \"216d4709-2b24-4351-bea6-b2d686dab302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:41.884659 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.884627    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c3fcc35a-jjmf2\" (UID: \"216d4709-2b24-4351-bea6-b2d686dab302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:41.884825 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.884755    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c3fcc35a-jjmf2\" (UID: \"216d4709-2b24-4351-bea6-b2d686dab302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:41.884825 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.884779    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nm5x\" (UniqueName: \"kubernetes.io/projected/216d4709-2b24-4351-bea6-b2d686dab302-kube-api-access-2nm5x\") pod \"maas-api-revoke-sub-c3fcc35a-jjmf2\" (UID: \"216d4709-2b24-4351-bea6-b2d686dab302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:41.885343 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.885309    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c3fcc35a-jjmf2\" (UID: \"216d4709-2b24-4351-bea6-b2d686dab302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:41.887141 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.887115    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c3fcc35a-jjmf2\" (UID: \"216d4709-2b24-4351-bea6-b2d686dab302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:41.900672 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.900648    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nm5x\" (UniqueName: \"kubernetes.io/projected/216d4709-2b24-4351-bea6-b2d686dab302-kube-api-access-2nm5x\") pod \"maas-api-revoke-sub-c3fcc35a-jjmf2\" (UID: \"216d4709-2b24-4351-bea6-b2d686dab302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:41.976343 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:41.976269    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:42.259640 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:42.259608    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"]
Nov 04 07:41:42.261039 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:41:42.261014    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216d4709_2b24_4351_bea6_b2d686dab302.slice/crio-3f5bc1dd68917a8f468b828404dc880f56d6d60916a506c9272439cb83680e72 WatchSource:0}: Error finding container 3f5bc1dd68917a8f468b828404dc880f56d6d60916a506c9272439cb83680e72: Status 404 returned error can't find the container with id 3f5bc1dd68917a8f468b828404dc880f56d6d60916a506c9272439cb83680e72
Nov 04 07:41:42.299546 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:42.299516    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2" event={"ID":"216d4709-2b24-4351-bea6-b2d686dab302","Type":"ContainerStarted","Data":"3f5bc1dd68917a8f468b828404dc880f56d6d60916a506c9272439cb83680e72"}
Nov 04 07:41:42.725312 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:42.725277    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6" podUID="980bbac5-cf4e-4bbd-9fd6-60bf29548e37" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:41:43.315052 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:43.314527    5626 generic.go:356] "Generic (PLEG): container finished" podID="216d4709-2b24-4351-bea6-b2d686dab302" containerID="3528f7a96f3af3bb2c8e8ddb17130bb75e773b4ed2601cf3ae36ef1d04b390e2" exitCode=0
Nov 04 07:41:43.315052 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:43.314601    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2" event={"ID":"216d4709-2b24-4351-bea6-b2d686dab302","Type":"ContainerDied","Data":"3528f7a96f3af3bb2c8e8ddb17130bb75e773b4ed2601cf3ae36ef1d04b390e2"}
Nov 04 07:41:43.321127 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:43.321101    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"]
Nov 04 07:41:43.321329 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:43.321306    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" podUID="fea08504-e401-4132-8b45-1f9b36646205" containerName="istio-proxy" containerID="cri-o://372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6" gracePeriod=30
Nov 04 07:41:44.476559 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:44.476535    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:44.617167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:44.617078    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-openshift-ca\") pod \"216d4709-2b24-4351-bea6-b2d686dab302\" (UID: \"216d4709-2b24-4351-bea6-b2d686dab302\") "
Nov 04 07:41:44.617167 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:44.617127    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/216d4709-2b24-4351-bea6-b2d686dab302-kube-api-access-2nm5x\" (UniqueName: \"kubernetes.io/projected/216d4709-2b24-4351-bea6-b2d686dab302-kube-api-access-2nm5x\") pod \"216d4709-2b24-4351-bea6-b2d686dab302\" (UID: \"216d4709-2b24-4351-bea6-b2d686dab302\") "
Nov 04 07:41:44.617413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:44.617183    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-xks-ca\") pod \"216d4709-2b24-4351-bea6-b2d686dab302\" (UID: \"216d4709-2b24-4351-bea6-b2d686dab302\") "
Nov 04 07:41:44.617512 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:44.617489    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-openshift-ca" pod "216d4709-2b24-4351-bea6-b2d686dab302" (UID: "216d4709-2b24-4351-bea6-b2d686dab302"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:44.617714 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:44.617692    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:44.619260 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:44.619238    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/216d4709-2b24-4351-bea6-b2d686dab302-kube-api-access-2nm5x" pod "216d4709-2b24-4351-bea6-b2d686dab302" (UID: "216d4709-2b24-4351-bea6-b2d686dab302"). InnerVolumeSpecName "kube-api-access-2nm5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:44.619620 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:44.619603    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-xks-ca" pod "216d4709-2b24-4351-bea6-b2d686dab302" (UID: "216d4709-2b24-4351-bea6-b2d686dab302"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:41:44.718341 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:44.718313    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nm5x\" (UniqueName: \"kubernetes.io/projected/216d4709-2b24-4351-bea6-b2d686dab302-kube-api-access-2nm5x\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:44.718341 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:44.718340    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/216d4709-2b24-4351-bea6-b2d686dab302-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:45.333131 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:45.333100    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2"
Nov 04 07:41:45.333312 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:45.333101    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3fcc35a-jjmf2" event={"ID":"216d4709-2b24-4351-bea6-b2d686dab302","Type":"ContainerDied","Data":"3f5bc1dd68917a8f468b828404dc880f56d6d60916a506c9272439cb83680e72"}
Nov 04 07:41:45.333312 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:45.333235    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f5bc1dd68917a8f468b828404dc880f56d6d60916a506c9272439cb83680e72"
Nov 04 07:41:46.277032 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:46.277000    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"]
Nov 04 07:41:47.539168 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:47.539136    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t" podUID="2702960b-b62a-4bfd-8457-13eb82b88043" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:41:48.587801 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.587742    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:41:48.671065 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.671026    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-kube-api-access-5g9cz\" (UniqueName: \"kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-kube-api-access-5g9cz\") pod \"fea08504-e401-4132-8b45-1f9b36646205\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") "
Nov 04 07:41:48.671256 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.671126    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-istio-token\" (UniqueName: \"kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-istio-token\") pod \"fea08504-e401-4132-8b45-1f9b36646205\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") "
Nov 04 07:41:48.671256 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.671188    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-certs\") pod \"fea08504-e401-4132-8b45-1f9b36646205\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") "
Nov 04 07:41:48.671256 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.671228    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-envoy\") pod \"fea08504-e401-4132-8b45-1f9b36646205\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") "
Nov 04 07:41:48.671712 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.671679    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-certs" pod "fea08504-e401-4132-8b45-1f9b36646205" (UID: "fea08504-e401-4132-8b45-1f9b36646205"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:48.671912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.671868    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fea08504-e401-4132-8b45-1f9b36646205-istiod-ca-cert" pod "fea08504-e401-4132-8b45-1f9b36646205" (UID: "fea08504-e401-4132-8b45-1f9b36646205"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:48.672033 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.671355    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fea08504-e401-4132-8b45-1f9b36646205-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fea08504-e401-4132-8b45-1f9b36646205-istiod-ca-cert\") pod \"fea08504-e401-4132-8b45-1f9b36646205\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") "
Nov 04 07:41:48.672033 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.671988    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fea08504-e401-4132-8b45-1f9b36646205-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fea08504-e401-4132-8b45-1f9b36646205-istio-podinfo\") pod \"fea08504-e401-4132-8b45-1f9b36646205\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") "
Nov 04 07:41:48.672160 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.672054    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-data\") pod \"fea08504-e401-4132-8b45-1f9b36646205\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") "
Nov 04 07:41:48.672160 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.672142    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-socket\") pod \"fea08504-e401-4132-8b45-1f9b36646205\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") "
Nov 04 07:41:48.672287 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.672258    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-credential-socket\") pod \"fea08504-e401-4132-8b45-1f9b36646205\" (UID: \"fea08504-e401-4132-8b45-1f9b36646205\") "
Nov 04 07:41:48.672455 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.672428    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-socket" pod "fea08504-e401-4132-8b45-1f9b36646205" (UID: "fea08504-e401-4132-8b45-1f9b36646205"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:48.672673 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.672630    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-credential-socket" pod "fea08504-e401-4132-8b45-1f9b36646205" (UID: "fea08504-e401-4132-8b45-1f9b36646205"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:48.672673 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.672644    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-data" pod "fea08504-e401-4132-8b45-1f9b36646205" (UID: "fea08504-e401-4132-8b45-1f9b36646205"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:48.673421 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.673392    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:48.673521 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.673444    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fea08504-e401-4132-8b45-1f9b36646205-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:48.673521 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.673459    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:48.673521 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.673474    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:48.673521 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.673499    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:48.673752 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.673515    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-kube-api-access-5g9cz" pod "fea08504-e401-4132-8b45-1f9b36646205" (UID: "fea08504-e401-4132-8b45-1f9b36646205"). InnerVolumeSpecName "kube-api-access-5g9cz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:48.673752 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.673641    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-istio-token" pod "fea08504-e401-4132-8b45-1f9b36646205" (UID: "fea08504-e401-4132-8b45-1f9b36646205"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:48.673862 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.673764    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-envoy" pod "fea08504-e401-4132-8b45-1f9b36646205" (UID: "fea08504-e401-4132-8b45-1f9b36646205"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:41:48.674208 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.674191    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fea08504-e401-4132-8b45-1f9b36646205-istio-podinfo" pod "fea08504-e401-4132-8b45-1f9b36646205" (UID: "fea08504-e401-4132-8b45-1f9b36646205"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:41:48.774295 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.774264    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fea08504-e401-4132-8b45-1f9b36646205-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:48.774295 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.774289    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g9cz\" (UniqueName: \"kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-kube-api-access-5g9cz\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:48.774295 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.774301    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fea08504-e401-4132-8b45-1f9b36646205-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:48.774535 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:48.774310    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fea08504-e401-4132-8b45-1f9b36646205-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:49.366236 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:49.366195    5626 generic.go:356] "Generic (PLEG): container finished" podID="fea08504-e401-4132-8b45-1f9b36646205" containerID="372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6" exitCode=0
Nov 04 07:41:49.366414 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:49.366258    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"
Nov 04 07:41:49.366414 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:49.366275    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" event={"ID":"fea08504-e401-4132-8b45-1f9b36646205","Type":"ContainerDied","Data":"372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6"}
Nov 04 07:41:49.366414 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:49.366330    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv" event={"ID":"fea08504-e401-4132-8b45-1f9b36646205","Type":"ContainerDied","Data":"95aa391314356df51ade86e3e8bf97c2c8ee7be1ad4fa35486cf3afff72a70e6"}
Nov 04 07:41:49.366414 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:49.366350    5626 scope.go:122] "RemoveContainer" containerID="372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6"
Nov 04 07:41:49.377988 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:49.377970    5626 scope.go:122] "RemoveContainer" containerID="1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea"
Nov 04 07:41:49.387834 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:49.387808    5626 scope.go:122] "RemoveContainer" containerID="372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6"
Nov 04 07:41:49.388173 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:49.388140    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6\": container with ID starting with 372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6 not found: ID does not exist" containerID="372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6"
Nov 04 07:41:49.388249 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:49.388183    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6"} err="failed to get container status \"372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6\": rpc error: code = NotFound desc = could not find container \"372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6\": container with ID starting with 372fdff2ca0c071825a05eb4b12ae9f18124870ff47dc40a39393ece3cf0d8b6 not found: ID does not exist"
Nov 04 07:41:49.388249 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:49.388199    5626 scope.go:122] "RemoveContainer" containerID="1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea"
Nov 04 07:41:49.388605 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:49.388578    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea\": container with ID starting with 1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea not found: ID does not exist" containerID="1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea"
Nov 04 07:41:49.388848 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:49.388609    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea"} err="failed to get container status \"1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea\": rpc error: code = NotFound desc = could not find container \"1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea\": container with ID starting with 1f56fec7263107889d74939562d17e8a5b09f0f98c5899b4b135a6429f1b87ea not found: ID does not exist"
Nov 04 07:41:49.394982 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:49.394959    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"]
Nov 04 07:41:49.398087 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:49.398062    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv"]
Nov 04 07:41:50.094326 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:50.094293    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fea08504-e401-4132-8b45-1f9b36646205" path="/var/lib/kubelet/pods/fea08504-e401-4132-8b45-1f9b36646205/volumes"
Nov 04 07:41:50.469895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:50.469813    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"]
Nov 04 07:41:52.366410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:52.366378    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"]
Nov 04 07:41:52.384813 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:52.384489    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"]
Nov 04 07:41:52.723476 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:41:52.723445    5626 logging.go:55] [core] [Channel #1814 SubChannel #1815]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.68:50052", ServerName: "10.134.0.68:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.68:50052: connect: connection refused"
Nov 04 07:41:52.899739 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:52.899712    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"
Nov 04 07:41:53.012980 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.012949    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"]
Nov 04 07:41:53.013555 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.013540    5626 cpu_manager.go:414] "containerMap: removing container" podUID="216d4709-2b24-4351-bea6-b2d686dab302" containerName="revoke-keys"
Nov 04 07:41:53.013555 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.013555    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="216d4709-2b24-4351-bea6-b2d686dab302" containerName="revoke-keys"
Nov 04 07:41:53.013654 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.013572    5626 cpu_manager.go:414] "containerMap: removing container" podUID="980bbac5-cf4e-4bbd-9fd6-60bf29548e37" containerName="payload-pre-processing"
Nov 04 07:41:53.013654 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.013578    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="980bbac5-cf4e-4bbd-9fd6-60bf29548e37" containerName="payload-pre-processing"
Nov 04 07:41:53.013654 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.013598    5626 cpu_manager.go:414] "containerMap: removing container" podUID="fea08504-e401-4132-8b45-1f9b36646205" containerName="istio-proxy"
Nov 04 07:41:53.013654 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.013603    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fea08504-e401-4132-8b45-1f9b36646205" containerName="istio-proxy"
Nov 04 07:41:53.013771 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.013702    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="fea08504-e401-4132-8b45-1f9b36646205" containerName="istio-proxy"
Nov 04 07:41:53.013771 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.013711    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="fea08504-e401-4132-8b45-1f9b36646205" containerName="istio-proxy"
Nov 04 07:41:53.013771 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.013724    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="216d4709-2b24-4351-bea6-b2d686dab302" containerName="revoke-keys"
Nov 04 07:41:53.013771 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.013731    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="980bbac5-cf4e-4bbd-9fd6-60bf29548e37" containerName="payload-pre-processing"
Nov 04 07:41:53.017279 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.017250    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:53.021826 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.021800    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:41:53.021826 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.021810    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:41:53.022117 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.022097    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:41:53.022117 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.022109    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:41:53.026379 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.026331    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"]
Nov 04 07:41:53.043877 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.043834    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-kube-api-access-xvkml\" (UniqueName: \"kubernetes.io/projected/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-kube-api-access-xvkml\") pod \"980bbac5-cf4e-4bbd-9fd6-60bf29548e37\" (UID: \"980bbac5-cf4e-4bbd-9fd6-60bf29548e37\") "
Nov 04 07:41:53.044002 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.043894    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-plugins-config-volume\") pod \"980bbac5-cf4e-4bbd-9fd6-60bf29548e37\" (UID: \"980bbac5-cf4e-4bbd-9fd6-60bf29548e37\") "
Nov 04 07:41:53.044392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.044351    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-plugins-config-volume" pod "980bbac5-cf4e-4bbd-9fd6-60bf29548e37" (UID: "980bbac5-cf4e-4bbd-9fd6-60bf29548e37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:53.044520 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.044490    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:53.046136 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.046113    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-kube-api-access-xvkml" pod "980bbac5-cf4e-4bbd-9fd6-60bf29548e37" (UID: "980bbac5-cf4e-4bbd-9fd6-60bf29548e37"). InnerVolumeSpecName "kube-api-access-xvkml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:53.145245 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.145156    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b3518b89-p2s5j\" (UID: \"a81809e4-7c2d-4c52-98a0-4e77f0124045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:53.145245 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.145192    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b3518b89-p2s5j\" (UID: \"a81809e4-7c2d-4c52-98a0-4e77f0124045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:53.145442 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.145402    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6r76\" (UniqueName: \"kubernetes.io/projected/a81809e4-7c2d-4c52-98a0-4e77f0124045-kube-api-access-s6r76\") pod \"maas-api-revoke-sub-b3518b89-p2s5j\" (UID: \"a81809e4-7c2d-4c52-98a0-4e77f0124045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:53.145557 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.145544    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvkml\" (UniqueName: \"kubernetes.io/projected/980bbac5-cf4e-4bbd-9fd6-60bf29548e37-kube-api-access-xvkml\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:53.246172 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.246139    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6r76\" (UniqueName: \"kubernetes.io/projected/a81809e4-7c2d-4c52-98a0-4e77f0124045-kube-api-access-s6r76\") pod \"maas-api-revoke-sub-b3518b89-p2s5j\" (UID: \"a81809e4-7c2d-4c52-98a0-4e77f0124045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:53.246340 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.246244    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b3518b89-p2s5j\" (UID: \"a81809e4-7c2d-4c52-98a0-4e77f0124045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:53.246340 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.246271    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b3518b89-p2s5j\" (UID: \"a81809e4-7c2d-4c52-98a0-4e77f0124045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:53.247208 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.247178    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b3518b89-p2s5j\" (UID: \"a81809e4-7c2d-4c52-98a0-4e77f0124045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:53.248726 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.248706    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b3518b89-p2s5j\" (UID: \"a81809e4-7c2d-4c52-98a0-4e77f0124045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:53.259881 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.259856    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6r76\" (UniqueName: \"kubernetes.io/projected/a81809e4-7c2d-4c52-98a0-4e77f0124045-kube-api-access-s6r76\") pod \"maas-api-revoke-sub-b3518b89-p2s5j\" (UID: \"a81809e4-7c2d-4c52-98a0-4e77f0124045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:53.342602 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.342564    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:53.408741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.408326    5626 generic.go:356] "Generic (PLEG): container finished" podID="980bbac5-cf4e-4bbd-9fd6-60bf29548e37" containerID="0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b" exitCode=0
Nov 04 07:41:53.408741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.408422    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"
Nov 04 07:41:53.408741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.408483    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6" event={"ID":"980bbac5-cf4e-4bbd-9fd6-60bf29548e37","Type":"ContainerDied","Data":"0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b"}
Nov 04 07:41:53.408741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.408532    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6" event={"ID":"980bbac5-cf4e-4bbd-9fd6-60bf29548e37","Type":"ContainerDied","Data":"05ca121a1309b465aa479c77fc184000bd39b70d0eb81def771aafd281afa4ac"}
Nov 04 07:41:53.408741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.408553    5626 scope.go:122] "RemoveContainer" containerID="0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b"
Nov 04 07:41:53.432655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.432630    5626 scope.go:122] "RemoveContainer" containerID="0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b"
Nov 04 07:41:53.433047 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:53.433017    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b\": container with ID starting with 0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b not found: ID does not exist" containerID="0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b"
Nov 04 07:41:53.433151 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.433059    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b"} err="failed to get container status \"0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b\": rpc error: code = NotFound desc = could not find container \"0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b\": container with ID starting with 0fa872e2578b104aa0f2fe22bf8983bc3ba2c3d4465e9b41db4c16b2489e8b6b not found: ID does not exist"
Nov 04 07:41:53.446986 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.446930    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"]
Nov 04 07:41:53.452341 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.452298    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6"]
Nov 04 07:41:53.661130 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.661069    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"]
Nov 04 07:41:53.723098 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:53.723058    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6" podUID="980bbac5-cf4e-4bbd-9fd6-60bf29548e37" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.68:50052\" within 1s: context deadline exceeded"
Nov 04 07:41:54.094413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:54.094340    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="980bbac5-cf4e-4bbd-9fd6-60bf29548e37" path="/var/lib/kubelet/pods/980bbac5-cf4e-4bbd-9fd6-60bf29548e37/volumes"
Nov 04 07:41:54.418255 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:54.418215    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j" event={"ID":"a81809e4-7c2d-4c52-98a0-4e77f0124045","Type":"ContainerStarted","Data":"e8fe6eb2d67ffe06cafb2b0b55bc49fff5bd187f151a46b311f496112eead7a0"}
Nov 04 07:41:54.418707 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:54.418277    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j" event={"ID":"a81809e4-7c2d-4c52-98a0-4e77f0124045","Type":"ContainerStarted","Data":"187bcfb8f9efe95e35c61f36bdce5d3d5b2e0d86f102fcc81cb740627fc3c864"}
Nov 04 07:41:54.440269 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:54.440218    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j" podStartSLOduration=2.440204951 podStartE2EDuration="2.440204951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:41:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:41:54.43606001 +0000 UTC m=+1536.458275271" watchObservedRunningTime="2026-11-04 07:41:54.440204951 +0000 UTC m=+1536.462420207"
Nov 04 07:41:55.429293 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:55.429256    5626 generic.go:356] "Generic (PLEG): container finished" podID="a81809e4-7c2d-4c52-98a0-4e77f0124045" containerID="e8fe6eb2d67ffe06cafb2b0b55bc49fff5bd187f151a46b311f496112eead7a0" exitCode=0
Nov 04 07:41:55.429722 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:55.429309    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j" event={"ID":"a81809e4-7c2d-4c52-98a0-4e77f0124045","Type":"ContainerDied","Data":"e8fe6eb2d67ffe06cafb2b0b55bc49fff5bd187f151a46b311f496112eead7a0"}
Nov 04 07:41:55.471047 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:55.470993    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb" podUID="aceda029-76df-4e8a-ba59-115b11c12b7f" containerName="payload-processing" containerID="cri-o://11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160" gracePeriod=25
Nov 04 07:41:55.816532 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:55.816091    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"
Nov 04 07:41:56.015380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:56.005853    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aceda029-76df-4e8a-ba59-115b11c12b7f-kube-api-access-k2bxr\" (UniqueName: \"kubernetes.io/projected/aceda029-76df-4e8a-ba59-115b11c12b7f-kube-api-access-k2bxr\") pod \"aceda029-76df-4e8a-ba59-115b11c12b7f\" (UID: \"aceda029-76df-4e8a-ba59-115b11c12b7f\") "
Nov 04 07:41:56.015380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:56.005969    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aceda029-76df-4e8a-ba59-115b11c12b7f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aceda029-76df-4e8a-ba59-115b11c12b7f-plugins-config-volume\") pod \"aceda029-76df-4e8a-ba59-115b11c12b7f\" (UID: \"aceda029-76df-4e8a-ba59-115b11c12b7f\") "
Nov 04 07:41:56.015380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:56.006791    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aceda029-76df-4e8a-ba59-115b11c12b7f-plugins-config-volume" pod "aceda029-76df-4e8a-ba59-115b11c12b7f" (UID: "aceda029-76df-4e8a-ba59-115b11c12b7f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:56.019937 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:56.019900    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aceda029-76df-4e8a-ba59-115b11c12b7f-kube-api-access-k2bxr" pod "aceda029-76df-4e8a-ba59-115b11c12b7f" (UID: "aceda029-76df-4e8a-ba59-115b11c12b7f"). InnerVolumeSpecName "kube-api-access-k2bxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:56.112439 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:56.111602    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aceda029-76df-4e8a-ba59-115b11c12b7f-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:56.112439 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:56.111638    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2bxr\" (UniqueName: \"kubernetes.io/projected/aceda029-76df-4e8a-ba59-115b11c12b7f-kube-api-access-k2bxr\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:56.459433 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:56.457801    5626 generic.go:356] "Generic (PLEG): container finished" podID="aceda029-76df-4e8a-ba59-115b11c12b7f" containerID="11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160" exitCode=0
Nov 04 07:41:56.459433 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:56.458134    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"
Nov 04 07:41:56.475638 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:56.475600    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb" event={"ID":"aceda029-76df-4e8a-ba59-115b11c12b7f","Type":"ContainerDied","Data":"11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160"}
Nov 04 07:41:56.475806 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:56.475674    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb" event={"ID":"aceda029-76df-4e8a-ba59-115b11c12b7f","Type":"ContainerDied","Data":"a44aff20b37182e095866d9874c33343cfbbce74bf5c167e0354f534529f4068"}
Nov 04 07:41:56.475806 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:56.475704    5626 scope.go:122] "RemoveContainer" containerID="11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160"
Nov 04 07:41:56.727442 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:56.724211    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"]
Nov 04 07:41:57.362582 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:57.315665    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" podUID="573aaebc-ca66-4bf2-84a5-2057a5e77b87" containerName="main" probeResult="failure" output="Get \"https://10.134.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:41:57.433385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:57.432586    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:41:57.655441 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:57.614674    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"]
Nov 04 07:41:57.680879 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:57.659597    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:41:57.803464 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:57.803431    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"]
Nov 04 07:41:57.913437 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:57.912769    5626 scope.go:122] "RemoveContainer" containerID="11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160"
Nov 04 07:41:57.929391 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:57.919608    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc" podUID="efa49e6f-5463-4724-a2b6-51882cb8b317" containerName="payload-pre-processing" containerID="cri-o://ce8b8de9dd46d7e59a5fa9209aa66e1507e11a2a929e9d128607ba5ff5d7ebd8" gracePeriod=25
Nov 04 07:41:57.929391 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:41:57.922808    5626 logging.go:55] [core] [Channel #1834 SubChannel #1837]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.74:50052", ServerName: "10.134.0.74:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.74:50052: connect: connection refused"
Nov 04 07:41:57.929391 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:57.922869    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh" podUID="7244c03f-1118-4bab-92ec-34894a8c9c5f" containerName="payload-processing" containerID="cri-o://6802f9ca4da9fdd0fe9e988f5593d92454c719b5057f43ba2be0162eb5b25c97" gracePeriod=25
Nov 04 07:41:57.967628 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:57.964585    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160\": container with ID starting with 11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160 not found: ID does not exist" containerID="11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160"
Nov 04 07:41:57.967628 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:57.964810    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160"} err="failed to get container status \"11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160\": rpc error: code = NotFound desc = could not find container \"11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160\": container with ID starting with 11b4af611968252e8e5b3d7308da1de7ff06a13a996654bb63d6af1508d88160 not found: ID does not exist"
Nov 04 07:41:57.985506 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:57.985424    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb"]
Nov 04 07:41:57.995815 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:57.995711    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:58.070020 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.046227    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-xks-ca\") pod \"a81809e4-7c2d-4c52-98a0-4e77f0124045\" (UID: \"a81809e4-7c2d-4c52-98a0-4e77f0124045\") "
Nov 04 07:41:58.070020 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.046285    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a81809e4-7c2d-4c52-98a0-4e77f0124045-kube-api-access-s6r76\" (UniqueName: \"kubernetes.io/projected/a81809e4-7c2d-4c52-98a0-4e77f0124045-kube-api-access-s6r76\") pod \"a81809e4-7c2d-4c52-98a0-4e77f0124045\" (UID: \"a81809e4-7c2d-4c52-98a0-4e77f0124045\") "
Nov 04 07:41:58.070020 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.046411    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-openshift-ca\") pod \"a81809e4-7c2d-4c52-98a0-4e77f0124045\" (UID: \"a81809e4-7c2d-4c52-98a0-4e77f0124045\") "
Nov 04 07:41:58.086816 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.086671    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-openshift-ca" pod "a81809e4-7c2d-4c52-98a0-4e77f0124045" (UID: "a81809e4-7c2d-4c52-98a0-4e77f0124045"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:58.197395 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.197338    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:58.323397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.315996    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Nov 04 07:41:58.325481 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.325444    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn" podUID="573aaebc-ca66-4bf2-84a5-2057a5e77b87" containerName="main" probeResult="failure" output="Get \"https://10.134.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:41:58.336777 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.336722    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a81809e4-7c2d-4c52-98a0-4e77f0124045-kube-api-access-s6r76" pod "a81809e4-7c2d-4c52-98a0-4e77f0124045" (UID: "a81809e4-7c2d-4c52-98a0-4e77f0124045"). InnerVolumeSpecName "kube-api-access-s6r76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:58.362844 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.338335    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aceda029-76df-4e8a-ba59-115b11c12b7f" path="/var/lib/kubelet/pods/aceda029-76df-4e8a-ba59-115b11c12b7f/volumes"
Nov 04 07:41:58.362844 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.351494    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-xks-ca" pod "a81809e4-7c2d-4c52-98a0-4e77f0124045" (UID: "a81809e4-7c2d-4c52-98a0-4e77f0124045"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:41:58.400461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.397747    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:41:58.413398 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.413348    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a81809e4-7c2d-4c52-98a0-4e77f0124045-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:58.413617 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.413596    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6r76\" (UniqueName: \"kubernetes.io/projected/a81809e4-7c2d-4c52-98a0-4e77f0124045-kube-api-access-s6r76\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:58.622401 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.607722    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 04 07:41:58.622401 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.608082    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 04 07:41:58.664387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.663620    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t" podUID="2702960b-b62a-4bfd-8457-13eb82b88043" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.74:50052\" within 1s: context deadline exceeded"
Nov 04 07:41:58.668384 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.665015    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:41:58.668384 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.665403    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:41:58.676387 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:41:58.671064    5626 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2702960b_b62a_4bfd_8457_13eb82b88043.slice/crio-aabdfcd4c12ce7ce6d3343867c9583795c4f69d1ea652a5cbd376bc9fcf4f84b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2702960b_b62a_4bfd_8457_13eb82b88043.slice/crio-conmon-aabdfcd4c12ce7ce6d3343867c9583795c4f69d1ea652a5cbd376bc9fcf4f84b.scope\": RecentStats: unable to find data in memory cache]"
Nov 04 07:41:58.717641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.717095    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:41:58.949458 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:58.921037    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn"
Nov 04 07:41:59.327124 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.327080    5626 generic.go:356] "Generic (PLEG): container finished" podID="2702960b-b62a-4bfd-8457-13eb82b88043" containerID="aabdfcd4c12ce7ce6d3343867c9583795c4f69d1ea652a5cbd376bc9fcf4f84b" exitCode=0
Nov 04 07:41:59.327439 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.327417    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t" event={"ID":"2702960b-b62a-4bfd-8457-13eb82b88043","Type":"ContainerDied","Data":"aabdfcd4c12ce7ce6d3343867c9583795c4f69d1ea652a5cbd376bc9fcf4f84b"}
Nov 04 07:41:59.327612 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.327596    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t" event={"ID":"2702960b-b62a-4bfd-8457-13eb82b88043","Type":"ContainerDied","Data":"d542cbb4a7f5252ccfc9ad40f3f688cdef2578fe858abd1180233d0c4b521ebf"}
Nov 04 07:41:59.327718 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.327706    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d542cbb4a7f5252ccfc9ad40f3f688cdef2578fe858abd1180233d0c4b521ebf"
Nov 04 07:41:59.334981 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.331755    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"
Nov 04 07:41:59.370387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.355522    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j"
Nov 04 07:41:59.370387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.356798    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b3518b89-p2s5j" event={"ID":"a81809e4-7c2d-4c52-98a0-4e77f0124045","Type":"ContainerDied","Data":"187bcfb8f9efe95e35c61f36bdce5d3d5b2e0d86f102fcc81cb740627fc3c864"}
Nov 04 07:41:59.370387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.356865    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="187bcfb8f9efe95e35c61f36bdce5d3d5b2e0d86f102fcc81cb740627fc3c864"
Nov 04 07:41:59.481321 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.475590    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2702960b-b62a-4bfd-8457-13eb82b88043-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2702960b-b62a-4bfd-8457-13eb82b88043-plugins-config-volume\") pod \"2702960b-b62a-4bfd-8457-13eb82b88043\" (UID: \"2702960b-b62a-4bfd-8457-13eb82b88043\") "
Nov 04 07:41:59.493016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.491503    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2702960b-b62a-4bfd-8457-13eb82b88043-kube-api-access-kcf5g\" (UniqueName: \"kubernetes.io/projected/2702960b-b62a-4bfd-8457-13eb82b88043-kube-api-access-kcf5g\") pod \"2702960b-b62a-4bfd-8457-13eb82b88043\" (UID: \"2702960b-b62a-4bfd-8457-13eb82b88043\") "
Nov 04 07:41:59.493016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.481939    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2702960b-b62a-4bfd-8457-13eb82b88043-plugins-config-volume" pod "2702960b-b62a-4bfd-8457-13eb82b88043" (UID: "2702960b-b62a-4bfd-8457-13eb82b88043"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:41:59.532760 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.531697    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2702960b-b62a-4bfd-8457-13eb82b88043-kube-api-access-kcf5g" pod "2702960b-b62a-4bfd-8457-13eb82b88043" (UID: "2702960b-b62a-4bfd-8457-13eb82b88043"). InnerVolumeSpecName "kube-api-access-kcf5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:41:59.541671 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.541638    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"]
Nov 04 07:41:59.542131 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.542096    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy" containerID="cri-o://e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350" gracePeriod=30
Nov 04 07:41:59.663698 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.663599    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2702960b-b62a-4bfd-8457-13eb82b88043-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:41:59.663698 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:41:59.663650    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcf5g\" (UniqueName: \"kubernetes.io/projected/2702960b-b62a-4bfd-8457-13eb82b88043-kube-api-access-kcf5g\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:00.346075 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.346042    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"
Nov 04 07:42:00.371921 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.371836    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"]
Nov 04 07:42:00.385342 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.385307    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t"]
Nov 04 07:42:00.403935 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.403895    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"]
Nov 04 07:42:00.404534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.404513    5626 cpu_manager.go:414] "containerMap: removing container" podUID="aceda029-76df-4e8a-ba59-115b11c12b7f" containerName="payload-processing"
Nov 04 07:42:00.404534 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.404530    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aceda029-76df-4e8a-ba59-115b11c12b7f" containerName="payload-processing"
Nov 04 07:42:00.404719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.404549    5626 cpu_manager.go:414] "containerMap: removing container" podUID="fea08504-e401-4132-8b45-1f9b36646205" containerName="istio-proxy"
Nov 04 07:42:00.404719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.404554    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fea08504-e401-4132-8b45-1f9b36646205" containerName="istio-proxy"
Nov 04 07:42:00.404719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.404564    5626 cpu_manager.go:414] "containerMap: removing container" podUID="2702960b-b62a-4bfd-8457-13eb82b88043" containerName="payload-pre-processing"
Nov 04 07:42:00.404719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.404569    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2702960b-b62a-4bfd-8457-13eb82b88043" containerName="payload-pre-processing"
Nov 04 07:42:00.404719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.404585    5626 cpu_manager.go:414] "containerMap: removing container" podUID="a81809e4-7c2d-4c52-98a0-4e77f0124045" containerName="revoke-keys"
Nov 04 07:42:00.404719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.404589    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a81809e4-7c2d-4c52-98a0-4e77f0124045" containerName="revoke-keys"
Nov 04 07:42:00.404719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.404674    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="aceda029-76df-4e8a-ba59-115b11c12b7f" containerName="payload-processing"
Nov 04 07:42:00.404719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.404685    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="2702960b-b62a-4bfd-8457-13eb82b88043" containerName="payload-pre-processing"
Nov 04 07:42:00.404719 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.404694    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="a81809e4-7c2d-4c52-98a0-4e77f0124045" containerName="revoke-keys"
Nov 04 07:42:00.408427 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.408406    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:00.425610 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.425297    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:42:00.425610 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.425509    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:42:00.425881 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.425860    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:42:00.425965 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.425934    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:42:00.428801 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.428775    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"]
Nov 04 07:42:00.583384 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.582548    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hphm7\" (UniqueName: \"kubernetes.io/projected/700217a7-2875-4c01-a556-f0741045b436-kube-api-access-hphm7\") pod \"maas-api-revoke-sub-a4bede4a-5gfvp\" (UID: \"700217a7-2875-4c01-a556-f0741045b436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:00.583384 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.582627    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/700217a7-2875-4c01-a556-f0741045b436-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4bede4a-5gfvp\" (UID: \"700217a7-2875-4c01-a556-f0741045b436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:00.583384 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.582670    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/700217a7-2875-4c01-a556-f0741045b436-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4bede4a-5gfvp\" (UID: \"700217a7-2875-4c01-a556-f0741045b436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:00.691469 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.690148    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hphm7\" (UniqueName: \"kubernetes.io/projected/700217a7-2875-4c01-a556-f0741045b436-kube-api-access-hphm7\") pod \"maas-api-revoke-sub-a4bede4a-5gfvp\" (UID: \"700217a7-2875-4c01-a556-f0741045b436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:00.691469 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.690225    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/700217a7-2875-4c01-a556-f0741045b436-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4bede4a-5gfvp\" (UID: \"700217a7-2875-4c01-a556-f0741045b436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:00.691469 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.690271    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/700217a7-2875-4c01-a556-f0741045b436-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4bede4a-5gfvp\" (UID: \"700217a7-2875-4c01-a556-f0741045b436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:00.694046 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.694015    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/700217a7-2875-4c01-a556-f0741045b436-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4bede4a-5gfvp\" (UID: \"700217a7-2875-4c01-a556-f0741045b436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:00.695151 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.695124    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/700217a7-2875-4c01-a556-f0741045b436-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4bede4a-5gfvp\" (UID: \"700217a7-2875-4c01-a556-f0741045b436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:00.719808 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.719778    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hphm7\" (UniqueName: \"kubernetes.io/projected/700217a7-2875-4c01-a556-f0741045b436-kube-api-access-hphm7\") pod \"maas-api-revoke-sub-a4bede4a-5gfvp\" (UID: \"700217a7-2875-4c01-a556-f0741045b436\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:00.726612 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:00.726582    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:01.084675 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.084640    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"]
Nov 04 07:42:01.088174 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:42:01.088131    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod700217a7_2875_4c01_a556_f0741045b436.slice/crio-3bfacd8b13fada63b0ae2b4ac8ed5c0a0a4a499885ac558aea599b64b7a71420 WatchSource:0}: Error finding container 3bfacd8b13fada63b0ae2b4ac8ed5c0a0a4a499885ac558aea599b64b7a71420: Status 404 returned error can't find the container with id 3bfacd8b13fada63b0ae2b4ac8ed5c0a0a4a499885ac558aea599b64b7a71420
Nov 04 07:42:01.298161 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.298096    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main" containerID="cri-o://59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684" gracePeriod=45
Nov 04 07:42:01.371657 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.371570    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp" event={"ID":"700217a7-2875-4c01-a556-f0741045b436","Type":"ContainerStarted","Data":"3bfacd8b13fada63b0ae2b4ac8ed5c0a0a4a499885ac558aea599b64b7a71420"}
Nov 04 07:42:01.563441 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.563405    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:42:01.602116 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.602082    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-kube-api-access-6g428\" (UniqueName: \"kubernetes.io/projected/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-kube-api-access-6g428\") pod \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") "
Nov 04 07:42:01.602296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.602219    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-dshm\") pod \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") "
Nov 04 07:42:01.602296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.602254    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-model-cache\") pod \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") "
Nov 04 07:42:01.602296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.602292    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tmp-dir\") pod \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") "
Nov 04 07:42:01.602509 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.602405    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tls-certs\") pod \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") "
Nov 04 07:42:01.602509 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.602446    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-home\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-home\") pod \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\" (UID: \"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880\") "
Nov 04 07:42:01.602609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.602549    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-model-cache" pod "b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" (UID: "b4b5d45c-0b31-4c2d-abd2-46c78c3a2880"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:01.602901 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.602749    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tmp-dir" pod "b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" (UID: "b4b5d45c-0b31-4c2d-abd2-46c78c3a2880"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:01.603024 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.602984    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-home" pod "b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" (UID: "b4b5d45c-0b31-4c2d-abd2-46c78c3a2880"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:01.603222 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.603202    5626 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-model-cache\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:01.603320 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.603225    5626 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tmp-dir\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:01.603320 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.603240    5626 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-home\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:01.605010 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.604985    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-kube-api-access-6g428" pod "b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" (UID: "b4b5d45c-0b31-4c2d-abd2-46c78c3a2880"). InnerVolumeSpecName "kube-api-access-6g428". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:01.605109 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.605043    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tls-certs" pod "b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" (UID: "b4b5d45c-0b31-4c2d-abd2-46c78c3a2880"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:42:01.605226 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.605205    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-dshm" pod "b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" (UID: "b4b5d45c-0b31-4c2d-abd2-46c78c3a2880"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:01.704662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.704616    5626 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-tls-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:01.704662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.704656    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g428\" (UniqueName: \"kubernetes.io/projected/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-kube-api-access-6g428\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:01.704853 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.704672    5626 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880-dshm\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:01.725197 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:01.725155    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg" podUID="cd960e9e-0f1d-44f8-9256-92da85ab0ac7" containerName="payload-processing" containerID="cri-o://6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801" gracePeriod=25
Nov 04 07:42:02.007234 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.007205    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"
Nov 04 07:42:02.094048 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.094010    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2702960b-b62a-4bfd-8457-13eb82b88043" path="/var/lib/kubelet/pods/2702960b-b62a-4bfd-8457-13eb82b88043/volumes"
Nov 04 07:42:02.108892 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.108866    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-plugins-config-volume\") pod \"cd960e9e-0f1d-44f8-9256-92da85ab0ac7\" (UID: \"cd960e9e-0f1d-44f8-9256-92da85ab0ac7\") "
Nov 04 07:42:02.109033 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.108951    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-kube-api-access-8xh88\" (UniqueName: \"kubernetes.io/projected/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-kube-api-access-8xh88\") pod \"cd960e9e-0f1d-44f8-9256-92da85ab0ac7\" (UID: \"cd960e9e-0f1d-44f8-9256-92da85ab0ac7\") "
Nov 04 07:42:02.110600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.109407    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-plugins-config-volume" pod "cd960e9e-0f1d-44f8-9256-92da85ab0ac7" (UID: "cd960e9e-0f1d-44f8-9256-92da85ab0ac7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:02.110600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.109636    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:02.112201 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.112173    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-kube-api-access-8xh88" pod "cd960e9e-0f1d-44f8-9256-92da85ab0ac7" (UID: "cd960e9e-0f1d-44f8-9256-92da85ab0ac7"). InnerVolumeSpecName "kube-api-access-8xh88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:02.211169 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.211088    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xh88\" (UniqueName: \"kubernetes.io/projected/cd960e9e-0f1d-44f8-9256-92da85ab0ac7-kube-api-access-8xh88\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:02.382040 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.382002    5626 generic.go:356] "Generic (PLEG): container finished" podID="700217a7-2875-4c01-a556-f0741045b436" containerID="006d339f21f5d3070b3fd6bd2bc20f0177fc71cee66aac0f770f78f9b4ff35c1" exitCode=0
Nov 04 07:42:02.382485 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.382093    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp" event={"ID":"700217a7-2875-4c01-a556-f0741045b436","Type":"ContainerDied","Data":"006d339f21f5d3070b3fd6bd2bc20f0177fc71cee66aac0f770f78f9b4ff35c1"}
Nov 04 07:42:02.383663 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.383636    5626 generic.go:356] "Generic (PLEG): container finished" podID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerID="59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684" exitCode=0
Nov 04 07:42:02.383789 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.383678    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"
Nov 04 07:42:02.383789 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.383719    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" event={"ID":"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880","Type":"ContainerDied","Data":"59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684"}
Nov 04 07:42:02.383789 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.383757    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5" event={"ID":"b4b5d45c-0b31-4c2d-abd2-46c78c3a2880","Type":"ContainerDied","Data":"df9dfdb970ecd87ab97f7464861e24e1468745a640f2db6ed886ac739774a1ca"}
Nov 04 07:42:02.383789 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.383783    5626 scope.go:122] "RemoveContainer" containerID="59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684"
Nov 04 07:42:02.385447 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.385423    5626 generic.go:356] "Generic (PLEG): container finished" podID="cd960e9e-0f1d-44f8-9256-92da85ab0ac7" containerID="6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801" exitCode=0
Nov 04 07:42:02.385539 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.385462    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"
Nov 04 07:42:02.385539 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.385487    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg" event={"ID":"cd960e9e-0f1d-44f8-9256-92da85ab0ac7","Type":"ContainerDied","Data":"6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801"}
Nov 04 07:42:02.385539 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.385523    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg" event={"ID":"cd960e9e-0f1d-44f8-9256-92da85ab0ac7","Type":"ContainerDied","Data":"07777870d119c4f812e6f003350b1b0b7906559a0e43e63091eb971f1e1dba9a"}
Nov 04 07:42:02.395875 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.395860    5626 scope.go:122] "RemoveContainer" containerID="59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684"
Nov 04 07:42:02.396169 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:02.396153    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684\": container with ID starting with 59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684 not found: ID does not exist" containerID="59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684"
Nov 04 07:42:02.396215 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.396185    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684"} err="failed to get container status \"59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684\": rpc error: code = NotFound desc = could not find container \"59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684\": container with ID starting with 59b00e93aefc415d87452e44075adf08577235d72892368401c4e69a8ef90684 not found: ID does not exist"
Nov 04 07:42:02.396215 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.396200    5626 scope.go:122] "RemoveContainer" containerID="6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801"
Nov 04 07:42:02.413991 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.413965    5626 scope.go:122] "RemoveContainer" containerID="6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801"
Nov 04 07:42:02.414345 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:02.414326    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801\": container with ID starting with 6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801 not found: ID does not exist" containerID="6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801"
Nov 04 07:42:02.414458 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.414353    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801"} err="failed to get container status \"6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801\": rpc error: code = NotFound desc = could not find container \"6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801\": container with ID starting with 6d230ad55954a23bd652b76c5611daf06c23ba7105687975df104ad3478d4801 not found: ID does not exist"
Nov 04 07:42:02.423992 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.423967    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"]
Nov 04 07:42:02.427441 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.427421    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/simulator-w2-016e6c-kserve-6cfd9fb94-mdbl5"]
Nov 04 07:42:02.438762 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.438741    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"]
Nov 04 07:42:02.446549 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.446520    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg"]
Nov 04 07:42:02.628409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.619387    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn" podUID="f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd" containerName="payload-pre-processing" containerID="cri-o://ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5" gracePeriod=25
Nov 04 07:42:02.917998 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:02.917969    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"
Nov 04 07:42:03.024724 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.024675    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-plugins-config-volume\") pod \"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd\" (UID: \"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd\") "
Nov 04 07:42:03.024911 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.024808    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-kube-api-access-r9pb4\" (UniqueName: \"kubernetes.io/projected/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-kube-api-access-r9pb4\") pod \"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd\" (UID: \"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd\") "
Nov 04 07:42:03.025141 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.025117    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-plugins-config-volume" pod "f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd" (UID: "f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:03.025234 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.025196    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:03.026954 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.026928    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-kube-api-access-r9pb4" pod "f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd" (UID: "f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd"). InnerVolumeSpecName "kube-api-access-r9pb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:03.126518 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.126478    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9pb4\" (UniqueName: \"kubernetes.io/projected/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd-kube-api-access-r9pb4\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:03.398210 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.398177    5626 generic.go:356] "Generic (PLEG): container finished" podID="f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd" containerID="ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5" exitCode=0
Nov 04 07:42:03.398693 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.398222    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"
Nov 04 07:42:03.398693 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.398272    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn" event={"ID":"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd","Type":"ContainerDied","Data":"ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5"}
Nov 04 07:42:03.398693 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.398332    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn" event={"ID":"f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd","Type":"ContainerDied","Data":"d6c886ac1966a399056ec38a806f51a471c48fc820e25848eb9e9ec4c100c9e8"}
Nov 04 07:42:03.398693 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.398396    5626 scope.go:122] "RemoveContainer" containerID="ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5"
Nov 04 07:42:03.413036 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.413008    5626 scope.go:122] "RemoveContainer" containerID="ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5"
Nov 04 07:42:03.413334 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:03.413310    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5\": container with ID starting with ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5 not found: ID does not exist" containerID="ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5"
Nov 04 07:42:03.413467 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.413345    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5"} err="failed to get container status \"ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5\": rpc error: code = NotFound desc = could not find container \"ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5\": container with ID starting with ec152d55365dbf182a161b1f05eda4bbed7d03bf253a82953a488e24a9c4c3d5 not found: ID does not exist"
Nov 04 07:42:03.418212 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.418185    5626 patch_prober.go:29] interesting pod/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 07:42:03.418518 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.418239    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 07:42:03.430397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.430350    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"]
Nov 04 07:42:03.437826 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.437800    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn"]
Nov 04 07:42:03.559809 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.559769    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:03.632413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.632374    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/700217a7-2875-4c01-a556-f0741045b436-kube-api-access-hphm7\" (UniqueName: \"kubernetes.io/projected/700217a7-2875-4c01-a556-f0741045b436-kube-api-access-hphm7\") pod \"700217a7-2875-4c01-a556-f0741045b436\" (UID: \"700217a7-2875-4c01-a556-f0741045b436\") "
Nov 04 07:42:03.632568 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.632472    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/700217a7-2875-4c01-a556-f0741045b436-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/700217a7-2875-4c01-a556-f0741045b436-maas-api-xks-ca\") pod \"700217a7-2875-4c01-a556-f0741045b436\" (UID: \"700217a7-2875-4c01-a556-f0741045b436\") "
Nov 04 07:42:03.632568 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.632521    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/700217a7-2875-4c01-a556-f0741045b436-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/700217a7-2875-4c01-a556-f0741045b436-maas-api-openshift-ca\") pod \"700217a7-2875-4c01-a556-f0741045b436\" (UID: \"700217a7-2875-4c01-a556-f0741045b436\") "
Nov 04 07:42:03.632926 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.632898    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/700217a7-2875-4c01-a556-f0741045b436-maas-api-openshift-ca" pod "700217a7-2875-4c01-a556-f0741045b436" (UID: "700217a7-2875-4c01-a556-f0741045b436"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:03.633166 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.633147    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/700217a7-2875-4c01-a556-f0741045b436-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:03.634662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.634632    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/700217a7-2875-4c01-a556-f0741045b436-kube-api-access-hphm7" pod "700217a7-2875-4c01-a556-f0741045b436" (UID: "700217a7-2875-4c01-a556-f0741045b436"). InnerVolumeSpecName "kube-api-access-hphm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:03.634761 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.634701    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/700217a7-2875-4c01-a556-f0741045b436-maas-api-xks-ca" pod "700217a7-2875-4c01-a556-f0741045b436" (UID: "700217a7-2875-4c01-a556-f0741045b436"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:42:03.734091 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.734047    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/700217a7-2875-4c01-a556-f0741045b436-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:03.734091 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:03.734089    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hphm7\" (UniqueName: \"kubernetes.io/projected/700217a7-2875-4c01-a556-f0741045b436-kube-api-access-hphm7\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:04.095284 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.095248    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" path="/var/lib/kubelet/pods/b4b5d45c-0b31-4c2d-abd2-46c78c3a2880/volumes"
Nov 04 07:42:04.095734 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.095718    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd960e9e-0f1d-44f8-9256-92da85ab0ac7" path="/var/lib/kubelet/pods/cd960e9e-0f1d-44f8-9256-92da85ab0ac7/volumes"
Nov 04 07:42:04.096099 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.096085    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd" path="/var/lib/kubelet/pods/f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd/volumes"
Nov 04 07:42:04.407917 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.407835    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp"
Nov 04 07:42:04.407917 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.407873    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4bede4a-5gfvp" event={"ID":"700217a7-2875-4c01-a556-f0741045b436","Type":"ContainerDied","Data":"3bfacd8b13fada63b0ae2b4ac8ed5c0a0a4a499885ac558aea599b64b7a71420"}
Nov 04 07:42:04.408556 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.407926    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bfacd8b13fada63b0ae2b4ac8ed5c0a0a4a499885ac558aea599b64b7a71420"
Nov 04 07:42:04.824871 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.824841    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:42:04.846987 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.846958    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-token\" (UniqueName: \"kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-token\") pod \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") "
Nov 04 07:42:04.847168 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847003    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-podinfo\") pod \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") "
Nov 04 07:42:04.847168 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847068    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istiod-ca-cert\") pod \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") "
Nov 04 07:42:04.847168 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847099    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-data\") pod \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") "
Nov 04 07:42:04.847168 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847147    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-kube-api-access-6vt8h\" (UniqueName: \"kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-kube-api-access-6vt8h\") pod \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") "
Nov 04 07:42:04.847416 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847219    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-socket\") pod \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") "
Nov 04 07:42:04.847416 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847254    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-credential-socket\") pod \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") "
Nov 04 07:42:04.847416 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847351    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-certs\") pod \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") "
Nov 04 07:42:04.847588 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847409    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-data" pod "c915aa8b-a06f-4446-a012-dd9c0f383b7f" (UID: "c915aa8b-a06f-4446-a012-dd9c0f383b7f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:04.847588 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847437    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-envoy\") pod \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\" (UID: \"c915aa8b-a06f-4446-a012-dd9c0f383b7f\") "
Nov 04 07:42:04.847588 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847559    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-certs" pod "c915aa8b-a06f-4446-a012-dd9c0f383b7f" (UID: "c915aa8b-a06f-4446-a012-dd9c0f383b7f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:04.847769 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847599    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istiod-ca-cert" pod "c915aa8b-a06f-4446-a012-dd9c0f383b7f" (UID: "c915aa8b-a06f-4446-a012-dd9c0f383b7f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:04.848248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847952    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:04.848248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847977    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:04.848248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.847993    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:04.848248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.848210    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-socket" pod "c915aa8b-a06f-4446-a012-dd9c0f383b7f" (UID: "c915aa8b-a06f-4446-a012-dd9c0f383b7f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:04.848248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.848228    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-credential-socket" pod "c915aa8b-a06f-4446-a012-dd9c0f383b7f" (UID: "c915aa8b-a06f-4446-a012-dd9c0f383b7f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:04.851429 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.851036    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-kube-api-access-6vt8h" pod "c915aa8b-a06f-4446-a012-dd9c0f383b7f" (UID: "c915aa8b-a06f-4446-a012-dd9c0f383b7f"). InnerVolumeSpecName "kube-api-access-6vt8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:04.851661 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.851633    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-podinfo" pod "c915aa8b-a06f-4446-a012-dd9c0f383b7f" (UID: "c915aa8b-a06f-4446-a012-dd9c0f383b7f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:42:04.851790 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.851692    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-token" pod "c915aa8b-a06f-4446-a012-dd9c0f383b7f" (UID: "c915aa8b-a06f-4446-a012-dd9c0f383b7f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:04.854819 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.854792    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-envoy" pod "c915aa8b-a06f-4446-a012-dd9c0f383b7f" (UID: "c915aa8b-a06f-4446-a012-dd9c0f383b7f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:04.940767 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.940673    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"]
Nov 04 07:42:04.949670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.949630    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:04.949670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.949659    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:04.949670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.949674    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c915aa8b-a06f-4446-a012-dd9c0f383b7f-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:04.949900 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.949686    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vt8h\" (UniqueName: \"kubernetes.io/projected/c915aa8b-a06f-4446-a012-dd9c0f383b7f-kube-api-access-6vt8h\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:04.949900 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.949697    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:04.949900 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.949705    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c915aa8b-a06f-4446-a012-dd9c0f383b7f-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:04.973752 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:04.973719    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"]
Nov 04 07:42:05.422238 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:05.422206    5626 generic.go:356] "Generic (PLEG): container finished" podID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerID="e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350" exitCode=0
Nov 04 07:42:05.422800 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:05.422262    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"
Nov 04 07:42:05.422800 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:05.422293    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" event={"ID":"c915aa8b-a06f-4446-a012-dd9c0f383b7f","Type":"ContainerDied","Data":"e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350"}
Nov 04 07:42:05.422800 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:05.422389    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc" event={"ID":"c915aa8b-a06f-4446-a012-dd9c0f383b7f","Type":"ContainerDied","Data":"a026b3110aea4d2ab8416acecd2082478f7115016e64be4a90f0cb61ab65a609"}
Nov 04 07:42:05.422800 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:05.422390    5626 scope.go:122] "RemoveContainer" containerID="e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350"
Nov 04 07:42:05.433856 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:05.433840    5626 scope.go:122] "RemoveContainer" containerID="1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8"
Nov 04 07:42:05.443604 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:05.443584    5626 scope.go:122] "RemoveContainer" containerID="e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350"
Nov 04 07:42:05.443876 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:05.443854    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350\": container with ID starting with e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350 not found: ID does not exist" containerID="e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350"
Nov 04 07:42:05.443958 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:05.443890    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350"} err="failed to get container status \"e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350\": rpc error: code = NotFound desc = could not find container \"e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350\": container with ID starting with e63f7273ba867a704bac9e69baa49bc5a929302555bc0619981608dc19941350 not found: ID does not exist"
Nov 04 07:42:05.443958 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:05.443915    5626 scope.go:122] "RemoveContainer" containerID="1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8"
Nov 04 07:42:05.444192 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:05.444174    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8\": container with ID starting with 1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8 not found: ID does not exist" containerID="1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8"
Nov 04 07:42:05.444233 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:05.444201    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8"} err="failed to get container status \"1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8\": rpc error: code = NotFound desc = could not find container \"1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8\": container with ID starting with 1451d5df32706072aef2797c9757f31edd223521cc2951c0cff1a78117830ee8 not found: ID does not exist"
Nov 04 07:42:05.449948 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:05.449871    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"]
Nov 04 07:42:05.457743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:05.457715    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc"]
Nov 04 07:42:06.093506 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:06.093475    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" path="/var/lib/kubelet/pods/c915aa8b-a06f-4446-a012-dd9c0f383b7f/volumes"
Nov 04 07:42:06.172460 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:06.172427    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:42:06.506046 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:06.505953    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc" podUID="efa49e6f-5463-4724-a2b6-51882cb8b317" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:42:06.506505 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:06.506351    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh" podUID="7244c03f-1118-4bab-92ec-34894a8c9c5f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:42:11.909209 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:11.909132    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 07:42:11.909760 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:11.909210    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:42:12.029005 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:12.028967    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"]
Nov 04 07:42:12.046411 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:12.046354    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"]
Nov 04 07:42:12.880821 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:12.880783    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"]
Nov 04 07:42:12.881105 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:12.881076    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy" containerID="cri-o://c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af" gracePeriod=30
Nov 04 07:42:13.125788 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.125749    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"]
Nov 04 07:42:13.126542 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126521    5626 cpu_manager.go:414] "containerMap: removing container" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy"
Nov 04 07:42:13.126624 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126544    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy"
Nov 04 07:42:13.126624 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126577    5626 cpu_manager.go:414] "containerMap: removing container" podUID="cd960e9e-0f1d-44f8-9256-92da85ab0ac7" containerName="payload-processing"
Nov 04 07:42:13.126624 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126586    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd960e9e-0f1d-44f8-9256-92da85ab0ac7" containerName="payload-processing"
Nov 04 07:42:13.126624 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126606    5626 cpu_manager.go:414] "containerMap: removing container" podUID="700217a7-2875-4c01-a556-f0741045b436" containerName="revoke-keys"
Nov 04 07:42:13.126624 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126614    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="700217a7-2875-4c01-a556-f0741045b436" containerName="revoke-keys"
Nov 04 07:42:13.126840 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126625    5626 cpu_manager.go:414] "containerMap: removing container" podUID="f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd" containerName="payload-pre-processing"
Nov 04 07:42:13.126840 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126632    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd" containerName="payload-pre-processing"
Nov 04 07:42:13.126840 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126658    5626 cpu_manager.go:414] "containerMap: removing container" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main"
Nov 04 07:42:13.126840 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126668    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main"
Nov 04 07:42:13.126840 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126798    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="700217a7-2875-4c01-a556-f0741045b436" containerName="revoke-keys"
Nov 04 07:42:13.126840 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126813    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9ef5cbc-8ec6-4414-a74e-afe5c784b0dd" containerName="payload-pre-processing"
Nov 04 07:42:13.126840 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126827    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="b4b5d45c-0b31-4c2d-abd2-46c78c3a2880" containerName="main"
Nov 04 07:42:13.126840 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126841    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy"
Nov 04 07:42:13.127078 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.126853    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd960e9e-0f1d-44f8-9256-92da85ab0ac7" containerName="payload-processing"
Nov 04 07:42:13.130632 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.130612    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:13.137386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.135698    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:42:13.137386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.135706    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:42:13.137386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.136618    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:42:13.137386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.136640    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:42:13.140903 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.140501    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"]
Nov 04 07:42:13.230506 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.230461    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d8e11c6e-2hvw5\" (UID: \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:13.230762 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.230736    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d8e11c6e-2hvw5\" (UID: \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:13.230949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.230924    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkdtc\" (UniqueName: \"kubernetes.io/projected/0e3d2c37-c22a-41d5-985b-6ab478e9528a-kube-api-access-jkdtc\") pod \"maas-api-revoke-sub-d8e11c6e-2hvw5\" (UID: \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:13.332067 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.332034    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d8e11c6e-2hvw5\" (UID: \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:13.332237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.332082    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkdtc\" (UniqueName: \"kubernetes.io/projected/0e3d2c37-c22a-41d5-985b-6ab478e9528a-kube-api-access-jkdtc\") pod \"maas-api-revoke-sub-d8e11c6e-2hvw5\" (UID: \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:13.332237 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.332200    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d8e11c6e-2hvw5\" (UID: \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:13.332846 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.332822    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d8e11c6e-2hvw5\" (UID: \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:13.334246 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.334224    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d8e11c6e-2hvw5\" (UID: \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:13.344720 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.344700    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkdtc\" (UniqueName: \"kubernetes.io/projected/0e3d2c37-c22a-41d5-985b-6ab478e9528a-kube-api-access-jkdtc\") pod \"maas-api-revoke-sub-d8e11c6e-2hvw5\" (UID: \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:13.448921 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.448839    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:13.733094 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:42:13.733062    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3d2c37_c22a_41d5_985b_6ab478e9528a.slice/crio-784600a66157ff003df6d78e088ca34a32d80e07330dd2f24cd6bcc77a230577 WatchSource:0}: Error finding container 784600a66157ff003df6d78e088ca34a32d80e07330dd2f24cd6bcc77a230577: Status 404 returned error can't find the container with id 784600a66157ff003df6d78e088ca34a32d80e07330dd2f24cd6bcc77a230577
Nov 04 07:42:13.733245 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:13.733171    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"]
Nov 04 07:42:14.599928 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:14.599897    5626 generic.go:356] "Generic (PLEG): container finished" podID="0e3d2c37-c22a-41d5-985b-6ab478e9528a" containerID="89d9dc7878d2fb3c897f408c380e41868aff958b1115c8b1362e63583638bd9d" exitCode=0
Nov 04 07:42:14.600414 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:14.599975    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5" event={"ID":"0e3d2c37-c22a-41d5-985b-6ab478e9528a","Type":"ContainerDied","Data":"89d9dc7878d2fb3c897f408c380e41868aff958b1115c8b1362e63583638bd9d"}
Nov 04 07:42:14.600414 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:14.600025    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5" event={"ID":"0e3d2c37-c22a-41d5-985b-6ab478e9528a","Type":"ContainerStarted","Data":"784600a66157ff003df6d78e088ca34a32d80e07330dd2f24cd6bcc77a230577"}
Nov 04 07:42:14.907946 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:14.907846    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"]
Nov 04 07:42:14.908490 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:14.908473    5626 cpu_manager.go:414] "containerMap: removing container" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy"
Nov 04 07:42:14.908543 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:14.908491    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy"
Nov 04 07:42:14.908622 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:14.908610    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="c915aa8b-a06f-4446-a012-dd9c0f383b7f" containerName="istio-proxy"
Nov 04 07:42:14.912039 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:14.912023    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:14.925711 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:14.925683    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"]
Nov 04 07:42:15.065335 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.065296    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff9f13c6-f606-4e5a-88dc-3987c2092418-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.065702 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.065341    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.065702 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.065519    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.065702 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.065565    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjbqc\" (UniqueName: \"kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-kube-api-access-rjbqc\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.065702 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.065596    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.065702 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.065643    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.065879 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.065725    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.065879 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.065783    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.065879 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.065816    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.167309 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167233    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff9f13c6-f606-4e5a-88dc-3987c2092418-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.167309 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167273    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.167573 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167318    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.167573 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167338    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjbqc\" (UniqueName: \"kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-kube-api-access-rjbqc\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.167573 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167391    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.167573 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167433    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.167573 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167461    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.167573 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167493    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.167573 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167533    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.167961 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167798    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.167961 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167939    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.167961 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167951    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.168128 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.167988    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.168180 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.168129    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff9f13c6-f606-4e5a-88dc-3987c2092418-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.170181 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.170161    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.170532 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.170508    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.185662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.185637    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.185849 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.185827    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjbqc\" (UniqueName: \"kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-kube-api-access-rjbqc\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.228231 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.228193    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:15.513956 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.513925    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"]
Nov 04 07:42:15.514544 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:42:15.514518    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9f13c6_f606_4e5a_88dc_3987c2092418.slice/crio-e3163fed070ea3a2a8e9543b9cf60ee9a84d11a0169c9c89fd357535c6309dd1 WatchSource:0}: Error finding container e3163fed070ea3a2a8e9543b9cf60ee9a84d11a0169c9c89fd357535c6309dd1: Status 404 returned error can't find the container with id e3163fed070ea3a2a8e9543b9cf60ee9a84d11a0169c9c89fd357535c6309dd1
Nov 04 07:42:15.516900 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.516862    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:42:15.517007 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.516940    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:42:15.517007 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.516987    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:42:15.609741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.609706    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" event={"ID":"ff9f13c6-f606-4e5a-88dc-3987c2092418","Type":"ContainerStarted","Data":"168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e"}
Nov 04 07:42:15.610093 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.609782    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" event={"ID":"ff9f13c6-f606-4e5a-88dc-3987c2092418","Type":"ContainerStarted","Data":"e3163fed070ea3a2a8e9543b9cf60ee9a84d11a0169c9c89fd357535c6309dd1"}
Nov 04 07:42:15.724403 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.720613    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" podStartSLOduration=1.720594593 podStartE2EDuration="1.720594593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:42:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:42:15.712589051 +0000 UTC m=+1557.734804324" watchObservedRunningTime="2026-11-04 07:42:15.720594593 +0000 UTC m=+1557.742809853"
Nov 04 07:42:15.767046 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.766984    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:15.876693 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.876666    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-xks-ca\") pod \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\" (UID: \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\") "
Nov 04 07:42:15.876872 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.876780    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e3d2c37-c22a-41d5-985b-6ab478e9528a-kube-api-access-jkdtc\" (UniqueName: \"kubernetes.io/projected/0e3d2c37-c22a-41d5-985b-6ab478e9528a-kube-api-access-jkdtc\") pod \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\" (UID: \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\") "
Nov 04 07:42:15.876872 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.876837    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-openshift-ca\") pod \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\" (UID: \"0e3d2c37-c22a-41d5-985b-6ab478e9528a\") "
Nov 04 07:42:15.877307 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.877258    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-openshift-ca" pod "0e3d2c37-c22a-41d5-985b-6ab478e9528a" (UID: "0e3d2c37-c22a-41d5-985b-6ab478e9528a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:15.877468 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.877403    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:15.879394 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.879349    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-xks-ca" pod "0e3d2c37-c22a-41d5-985b-6ab478e9528a" (UID: "0e3d2c37-c22a-41d5-985b-6ab478e9528a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:42:15.879527 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.879430    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e3d2c37-c22a-41d5-985b-6ab478e9528a-kube-api-access-jkdtc" pod "0e3d2c37-c22a-41d5-985b-6ab478e9528a" (UID: "0e3d2c37-c22a-41d5-985b-6ab478e9528a"). InnerVolumeSpecName "kube-api-access-jkdtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:15.988426 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.980296    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0e3d2c37-c22a-41d5-985b-6ab478e9528a-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:15.988426 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:15.980345    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkdtc\" (UniqueName: \"kubernetes.io/projected/0e3d2c37-c22a-41d5-985b-6ab478e9528a-kube-api-access-jkdtc\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:16.228929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:16.228897    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:16.230660 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:16.230633    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Nov 04 07:42:16.230777 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:16.230680    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" podUID="ff9f13c6-f606-4e5a-88dc-3987c2092418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Nov 04 07:42:16.504250 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:16.504159    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc" podUID="efa49e6f-5463-4724-a2b6-51882cb8b317" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:42:16.504804 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:16.504762    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh" podUID="7244c03f-1118-4bab-92ec-34894a8c9c5f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:42:16.619297 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:16.619268    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5"
Nov 04 07:42:16.619682 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:16.619656    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8e11c6e-2hvw5" event={"ID":"0e3d2c37-c22a-41d5-985b-6ab478e9528a","Type":"ContainerDied","Data":"784600a66157ff003df6d78e088ca34a32d80e07330dd2f24cd6bcc77a230577"}
Nov 04 07:42:16.619728 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:16.619690    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="784600a66157ff003df6d78e088ca34a32d80e07330dd2f24cd6bcc77a230577"
Nov 04 07:42:17.029578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:17.029527    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b" podUID="3477aa8d-a28d-4e96-8a17-1f177c1080ff" containerName="payload-processing" containerID="cri-o://ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23" gracePeriod=25
Nov 04 07:42:17.062452 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:17.051428    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv" podUID="9661dd1c-40ae-420f-96cf-af9a01477333" containerName="payload-pre-processing" containerID="cri-o://bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62" gracePeriod=25
Nov 04 07:42:17.228685 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:17.228653    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Nov 04 07:42:17.228854 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:17.228703    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" podUID="ff9f13c6-f606-4e5a-88dc-3987c2092418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Nov 04 07:42:18.228748 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.228720    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Nov 04 07:42:18.229141 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.229118    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" podUID="ff9f13c6-f606-4e5a-88dc-3987c2092418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Nov 04 07:42:18.309253 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.309218    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:42:18.335348 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.335308    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"
Nov 04 07:42:18.370560 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.370529    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"
Nov 04 07:42:18.411482 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.411401    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-podinfo\") pod \"222aacaf-0c37-4aad-91ad-5607a50820ce\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") "
Nov 04 07:42:18.411482 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.411441    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-kube-api-access-nmf4h\" (UniqueName: \"kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-kube-api-access-nmf4h\") pod \"222aacaf-0c37-4aad-91ad-5607a50820ce\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") "
Nov 04 07:42:18.411664 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.411485    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-token\" (UniqueName: \"kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-token\") pod \"222aacaf-0c37-4aad-91ad-5607a50820ce\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") "
Nov 04 07:42:18.411664 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.411548    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/222aacaf-0c37-4aad-91ad-5607a50820ce-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/222aacaf-0c37-4aad-91ad-5607a50820ce-istiod-ca-cert\") pod \"222aacaf-0c37-4aad-91ad-5607a50820ce\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") "
Nov 04 07:42:18.411664 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.411575    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-credential-socket\") pod \"222aacaf-0c37-4aad-91ad-5607a50820ce\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") "
Nov 04 07:42:18.411664 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.411597    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-socket\") pod \"222aacaf-0c37-4aad-91ad-5607a50820ce\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") "
Nov 04 07:42:18.411664 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.411657    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-data\") pod \"222aacaf-0c37-4aad-91ad-5607a50820ce\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") "
Nov 04 07:42:18.411963 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.411722    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-envoy\") pod \"222aacaf-0c37-4aad-91ad-5607a50820ce\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") "
Nov 04 07:42:18.411963 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.411778    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-certs\") pod \"222aacaf-0c37-4aad-91ad-5607a50820ce\" (UID: \"222aacaf-0c37-4aad-91ad-5607a50820ce\") "
Nov 04 07:42:18.412075 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.411974    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-socket" pod "222aacaf-0c37-4aad-91ad-5607a50820ce" (UID: "222aacaf-0c37-4aad-91ad-5607a50820ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:18.412164 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.412041    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/222aacaf-0c37-4aad-91ad-5607a50820ce-istiod-ca-cert" pod "222aacaf-0c37-4aad-91ad-5607a50820ce" (UID: "222aacaf-0c37-4aad-91ad-5607a50820ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:18.412227 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.412144    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-data" pod "222aacaf-0c37-4aad-91ad-5607a50820ce" (UID: "222aacaf-0c37-4aad-91ad-5607a50820ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:18.412227 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.412197    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-certs" pod "222aacaf-0c37-4aad-91ad-5607a50820ce" (UID: "222aacaf-0c37-4aad-91ad-5607a50820ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:18.412227 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.412213    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-credential-socket" pod "222aacaf-0c37-4aad-91ad-5607a50820ce" (UID: "222aacaf-0c37-4aad-91ad-5607a50820ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:18.412452 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.412402    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/222aacaf-0c37-4aad-91ad-5607a50820ce-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.412452 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.412426    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.412452 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.412442    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.412619 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.412456    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.412619 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.412471    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.414085 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.414064    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-kube-api-access-nmf4h" pod "222aacaf-0c37-4aad-91ad-5607a50820ce" (UID: "222aacaf-0c37-4aad-91ad-5607a50820ce"). InnerVolumeSpecName "kube-api-access-nmf4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:18.414570 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.414547    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-envoy" pod "222aacaf-0c37-4aad-91ad-5607a50820ce" (UID: "222aacaf-0c37-4aad-91ad-5607a50820ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:18.414673 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.414613    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-token" pod "222aacaf-0c37-4aad-91ad-5607a50820ce" (UID: "222aacaf-0c37-4aad-91ad-5607a50820ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:18.414794 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.414771    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-podinfo" pod "222aacaf-0c37-4aad-91ad-5607a50820ce" (UID: "222aacaf-0c37-4aad-91ad-5607a50820ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:42:18.473896 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.473868    5626 scope.go:122] "RemoveContainer" containerID="6802f9ca4da9fdd0fe9e988f5593d92454c719b5057f43ba2be0162eb5b25c97"
Nov 04 07:42:18.483614 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.483594    5626 scope.go:122] "RemoveContainer" containerID="ce8b8de9dd46d7e59a5fa9209aa66e1507e11a2a929e9d128607ba5ff5d7ebd8"
Nov 04 07:42:18.513612 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.513588    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efa49e6f-5463-4724-a2b6-51882cb8b317-kube-api-access-xknsb\" (UniqueName: \"kubernetes.io/projected/efa49e6f-5463-4724-a2b6-51882cb8b317-kube-api-access-xknsb\") pod \"efa49e6f-5463-4724-a2b6-51882cb8b317\" (UID: \"efa49e6f-5463-4724-a2b6-51882cb8b317\") "
Nov 04 07:42:18.513689 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.513645    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7244c03f-1118-4bab-92ec-34894a8c9c5f-kube-api-access-mz4m9\" (UniqueName: \"kubernetes.io/projected/7244c03f-1118-4bab-92ec-34894a8c9c5f-kube-api-access-mz4m9\") pod \"7244c03f-1118-4bab-92ec-34894a8c9c5f\" (UID: \"7244c03f-1118-4bab-92ec-34894a8c9c5f\") "
Nov 04 07:42:18.513779 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.513764    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efa49e6f-5463-4724-a2b6-51882cb8b317-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efa49e6f-5463-4724-a2b6-51882cb8b317-plugins-config-volume\") pod \"efa49e6f-5463-4724-a2b6-51882cb8b317\" (UID: \"efa49e6f-5463-4724-a2b6-51882cb8b317\") "
Nov 04 07:42:18.513891 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.513875    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7244c03f-1118-4bab-92ec-34894a8c9c5f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7244c03f-1118-4bab-92ec-34894a8c9c5f-plugins-config-volume\") pod \"7244c03f-1118-4bab-92ec-34894a8c9c5f\" (UID: \"7244c03f-1118-4bab-92ec-34894a8c9c5f\") "
Nov 04 07:42:18.514176 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.514153    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa49e6f-5463-4724-a2b6-51882cb8b317-plugins-config-volume" pod "efa49e6f-5463-4724-a2b6-51882cb8b317" (UID: "efa49e6f-5463-4724-a2b6-51882cb8b317"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:18.514274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.514215    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7244c03f-1118-4bab-92ec-34894a8c9c5f-plugins-config-volume" pod "7244c03f-1118-4bab-92ec-34894a8c9c5f" (UID: "7244c03f-1118-4bab-92ec-34894a8c9c5f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:18.514421 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.514405    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efa49e6f-5463-4724-a2b6-51882cb8b317-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.514503 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.514427    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.514503 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.514442    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7244c03f-1118-4bab-92ec-34894a8c9c5f-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.514503 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.514457    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.514503 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.514475    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmf4h\" (UniqueName: \"kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-kube-api-access-nmf4h\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.514503 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.514488    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/222aacaf-0c37-4aad-91ad-5607a50820ce-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.515570 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.515551    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa49e6f-5463-4724-a2b6-51882cb8b317-kube-api-access-xknsb" pod "efa49e6f-5463-4724-a2b6-51882cb8b317" (UID: "efa49e6f-5463-4724-a2b6-51882cb8b317"). InnerVolumeSpecName "kube-api-access-xknsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:18.516306 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.516286    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7244c03f-1118-4bab-92ec-34894a8c9c5f-kube-api-access-mz4m9" pod "7244c03f-1118-4bab-92ec-34894a8c9c5f" (UID: "7244c03f-1118-4bab-92ec-34894a8c9c5f"). InnerVolumeSpecName "kube-api-access-mz4m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:18.615545 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.615518    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xknsb\" (UniqueName: \"kubernetes.io/projected/efa49e6f-5463-4724-a2b6-51882cb8b317-kube-api-access-xknsb\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.615545 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.615544    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz4m9\" (UniqueName: \"kubernetes.io/projected/7244c03f-1118-4bab-92ec-34894a8c9c5f-kube-api-access-mz4m9\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:18.633652 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.633626    5626 generic.go:356] "Generic (PLEG): container finished" podID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerID="c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af" exitCode=0
Nov 04 07:42:18.633772 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.633676    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"
Nov 04 07:42:18.633772 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.633712    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" event={"ID":"222aacaf-0c37-4aad-91ad-5607a50820ce","Type":"ContainerDied","Data":"c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af"}
Nov 04 07:42:18.633772 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.633759    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn" event={"ID":"222aacaf-0c37-4aad-91ad-5607a50820ce","Type":"ContainerDied","Data":"84321167d6f661a25828aa83b1bacbaea0a8fd2ae53c8e93c1ba2c6e4865ab27"}
Nov 04 07:42:18.633912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.633780    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh" event={"ID":"7244c03f-1118-4bab-92ec-34894a8c9c5f","Type":"ContainerDied","Data":"6802f9ca4da9fdd0fe9e988f5593d92454c719b5057f43ba2be0162eb5b25c97"}
Nov 04 07:42:18.633912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.633800    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh" event={"ID":"7244c03f-1118-4bab-92ec-34894a8c9c5f","Type":"ContainerDied","Data":"9e3b6b6611a956cb388ed1fbf46dcc4bd30e59168cc8536b357334329e7d1eb4"}
Nov 04 07:42:18.633912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.633819    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc" event={"ID":"efa49e6f-5463-4724-a2b6-51882cb8b317","Type":"ContainerDied","Data":"ce8b8de9dd46d7e59a5fa9209aa66e1507e11a2a929e9d128607ba5ff5d7ebd8"}
Nov 04 07:42:18.633912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.633838    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc" event={"ID":"efa49e6f-5463-4724-a2b6-51882cb8b317","Type":"ContainerDied","Data":"e3c902f0d0484f4244a8949ca730f16ab462753081bd2425e34660eab426623e"}
Nov 04 07:42:18.633912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.633841    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"
Nov 04 07:42:18.633912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.633842    5626 scope.go:122] "RemoveContainer" containerID="c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af"
Nov 04 07:42:18.634190 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.634001    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"
Nov 04 07:42:18.671858 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.671827    5626 scope.go:122] "RemoveContainer" containerID="2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087"
Nov 04 07:42:18.682156 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.682137    5626 scope.go:122] "RemoveContainer" containerID="c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af"
Nov 04 07:42:18.682519 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:18.682495    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af\": container with ID starting with c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af not found: ID does not exist" containerID="c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af"
Nov 04 07:42:18.682616 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.682532    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af"} err="failed to get container status \"c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af\": rpc error: code = NotFound desc = could not find container \"c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af\": container with ID starting with c9bf5ab24e7b11d8dbacfa3507f8e062175a25a5ce15b46900b09a3d267dc7af not found: ID does not exist"
Nov 04 07:42:18.682616 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.682554    5626 scope.go:122] "RemoveContainer" containerID="2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087"
Nov 04 07:42:18.682817 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:18.682800    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087\": container with ID starting with 2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087 not found: ID does not exist" containerID="2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087"
Nov 04 07:42:18.682876 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.682826    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087"} err="failed to get container status \"2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087\": rpc error: code = NotFound desc = could not find container \"2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087\": container with ID starting with 2b291e397a65f8c8ee8f2e88584cb69026dc29a0d08756459c806c0ef3438087 not found: ID does not exist"
Nov 04 07:42:18.714310 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.714287    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"]
Nov 04 07:42:18.717663 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.717643    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc"]
Nov 04 07:42:18.730761 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.730736    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"]
Nov 04 07:42:18.735165 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.735146    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn"]
Nov 04 07:42:18.766454 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.766407    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"]
Nov 04 07:42:18.770550 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:18.770521    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh"]
Nov 04 07:42:19.229420 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:19.229354    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Nov 04 07:42:19.229822 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:19.229469    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" podUID="ff9f13c6-f606-4e5a-88dc-3987c2092418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Nov 04 07:42:19.830623 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:19.830583    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"]
Nov 04 07:42:19.846386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:19.842210    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"]
Nov 04 07:42:19.973593 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:19.973530    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main" containerID="cri-o://f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097" gracePeriod=45
Nov 04 07:42:19.994634 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:19.994590    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" containerID="cri-o://3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc" gracePeriod=45
Nov 04 07:42:20.094797 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.094723    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" path="/var/lib/kubelet/pods/222aacaf-0c37-4aad-91ad-5607a50820ce/volumes"
Nov 04 07:42:20.095159 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.095144    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7244c03f-1118-4bab-92ec-34894a8c9c5f" path="/var/lib/kubelet/pods/7244c03f-1118-4bab-92ec-34894a8c9c5f/volumes"
Nov 04 07:42:20.095513 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.095501    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efa49e6f-5463-4724-a2b6-51882cb8b317" path="/var/lib/kubelet/pods/efa49e6f-5463-4724-a2b6-51882cb8b317/volumes"
Nov 04 07:42:20.230330 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.229909    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Nov 04 07:42:20.230330 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.229976    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" podUID="ff9f13c6-f606-4e5a-88dc-3987c2092418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Nov 04 07:42:20.495953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.495925    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:42:20.500251 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.500178    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.730772    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-model-cache\") pod \"cf8055f2-ea85-4b51-a7ce-766f09b12126\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") "
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.730840    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-tmp-dir\") pod \"cf8055f2-ea85-4b51-a7ce-766f09b12126\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") "
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.730882    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tmp-dir\") pod \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") "
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.730920    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-kube-api-access-mzgft\" (UniqueName: \"kubernetes.io/projected/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-kube-api-access-mzgft\") pod \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") "
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.730953    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf8055f2-ea85-4b51-a7ce-766f09b12126-tls-certs\" (UniqueName: \"kubernetes.io/secret/cf8055f2-ea85-4b51-a7ce-766f09b12126-tls-certs\") pod \"cf8055f2-ea85-4b51-a7ce-766f09b12126\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") "
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.730995    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tls-certs\" (UniqueName: \"kubernetes.io/secret/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tls-certs\") pod \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") "
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.731054    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf8055f2-ea85-4b51-a7ce-766f09b12126-kube-api-access-qg2xb\" (UniqueName: \"kubernetes.io/projected/cf8055f2-ea85-4b51-a7ce-766f09b12126-kube-api-access-qg2xb\") pod \"cf8055f2-ea85-4b51-a7ce-766f09b12126\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") "
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.731088    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-dshm\") pod \"cf8055f2-ea85-4b51-a7ce-766f09b12126\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") "
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.731165    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-dshm\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-dshm\") pod \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") "
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.731219    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-home\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-home\") pod \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") "
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.731267    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-model-cache\") pod \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\" (UID: \"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd\") "
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.731346    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-home\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-home\") pod \"cf8055f2-ea85-4b51-a7ce-766f09b12126\" (UID: \"cf8055f2-ea85-4b51-a7ce-766f09b12126\") "
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.732007    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-home" pod "cf8055f2-ea85-4b51-a7ce-766f09b12126" (UID: "cf8055f2-ea85-4b51-a7ce-766f09b12126"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.732195    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-model-cache" pod "cf8055f2-ea85-4b51-a7ce-766f09b12126" (UID: "cf8055f2-ea85-4b51-a7ce-766f09b12126"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.732394    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-tmp-dir" pod "cf8055f2-ea85-4b51-a7ce-766f09b12126" (UID: "cf8055f2-ea85-4b51-a7ce-766f09b12126"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:20.732997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.732587    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tmp-dir" pod "24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" (UID: "24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:20.734018 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.733718    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-home" pod "24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" (UID: "24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:20.734230 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.734200    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-model-cache" pod "24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" (UID: "24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:20.736929 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.736808    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf8055f2-ea85-4b51-a7ce-766f09b12126-tls-certs" pod "cf8055f2-ea85-4b51-a7ce-766f09b12126" (UID: "cf8055f2-ea85-4b51-a7ce-766f09b12126"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:42:20.796386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.777668    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-dshm" pod "24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" (UID: "24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:20.796386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.779110    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tls-certs" pod "24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" (UID: "24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:42:20.796386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.779225    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-dshm" pod "cf8055f2-ea85-4b51-a7ce-766f09b12126" (UID: "cf8055f2-ea85-4b51-a7ce-766f09b12126"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:20.796386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.781724    5626 generic.go:356] "Generic (PLEG): container finished" podID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerID="3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc" exitCode=0
Nov 04 07:42:20.796386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.781876    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" event={"ID":"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd","Type":"ContainerDied","Data":"3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc"}
Nov 04 07:42:20.796386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.781917    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" event={"ID":"24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd","Type":"ContainerDied","Data":"ff40c9b996d1c98fe8b72e151ddfca218a648c90cef3c4ac1c49139b9c16f6ce"}
Nov 04 07:42:20.796386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.781940    5626 scope.go:122] "RemoveContainer" containerID="3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc"
Nov 04 07:42:20.796386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.782091    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"
Nov 04 07:42:20.796386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.789512    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf8055f2-ea85-4b51-a7ce-766f09b12126-kube-api-access-qg2xb" pod "cf8055f2-ea85-4b51-a7ce-766f09b12126" (UID: "cf8055f2-ea85-4b51-a7ce-766f09b12126"). InnerVolumeSpecName "kube-api-access-qg2xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:20.796386 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.791103    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-kube-api-access-mzgft" pod "24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" (UID: "24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd"). InnerVolumeSpecName "kube-api-access-mzgft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:20.805410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.805315    5626 generic.go:356] "Generic (PLEG): container finished" podID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerID="f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097" exitCode=0
Nov 04 07:42:20.805553 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.805467    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" event={"ID":"cf8055f2-ea85-4b51-a7ce-766f09b12126","Type":"ContainerDied","Data":"f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097"}
Nov 04 07:42:20.805553 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.805529    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx" event={"ID":"cf8055f2-ea85-4b51-a7ce-766f09b12126","Type":"ContainerDied","Data":"38befc5c795fed62b5fb8037b5028713cf23b69c27f6fcd21868f5622ec945c3"}
Nov 04 07:42:20.805685 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.805629    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"
Nov 04 07:42:20.821821 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.821801    5626 scope.go:122] "RemoveContainer" containerID="3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc"
Nov 04 07:42:20.822084 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:20.822065    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc\": container with ID starting with 3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc not found: ID does not exist" containerID="3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc"
Nov 04 07:42:20.822157 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.822092    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc"} err="failed to get container status \"3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc\": rpc error: code = NotFound desc = could not find container \"3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc\": container with ID starting with 3a2406ab810615945a54e555cc15c879fa6d99a8047b287d2ac9bdeec8e1bfdc not found: ID does not exist"
Nov 04 07:42:20.822157 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.822109    5626 scope.go:122] "RemoveContainer" containerID="f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097"
Nov 04 07:42:20.832811 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.832788    5626 scope.go:122] "RemoveContainer" containerID="f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097"
Nov 04 07:42:20.832955 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.832934    5626 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-model-cache\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:20.833028 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.832962    5626 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-tmp-dir\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:20.833028 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.832978    5626 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tmp-dir\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:20.833028 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.832992    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzgft\" (UniqueName: \"kubernetes.io/projected/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-kube-api-access-mzgft\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:20.833028 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.833007    5626 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf8055f2-ea85-4b51-a7ce-766f09b12126-tls-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:20.833028 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.833022    5626 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-tls-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:20.833243 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.833035    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg2xb\" (UniqueName: \"kubernetes.io/projected/cf8055f2-ea85-4b51-a7ce-766f09b12126-kube-api-access-qg2xb\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:20.833243 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.833049    5626 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-dshm\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:20.833243 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.833062    5626 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-dshm\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:20.833243 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.833074    5626 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-home\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:20.833243 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.833084    5626 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd-model-cache\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:20.833243 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.833093    5626 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf8055f2-ea85-4b51-a7ce-766f09b12126-home\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:20.833243 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:20.833112    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097\": container with ID starting with f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097 not found: ID does not exist" containerID="f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097"
Nov 04 07:42:20.833243 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.833152    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097"} err="failed to get container status \"f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097\": rpc error: code = NotFound desc = could not find container \"f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097\": container with ID starting with f0310143d9c3a60f24fc6b2b64b7d5991325ab40a0027ce955a7cdc828e6d097 not found: ID does not exist"
Nov 04 07:42:20.843272 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.842574    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"]
Nov 04 07:42:20.869768 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:20.869733    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-df3be8/distinct-2-w0-df3be8-kserve-7bb8b7c9b4-q2xlx"]
Nov 04 07:42:21.114037 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:21.114003    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"]
Nov 04 07:42:21.119210 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:21.119170    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25"]
Nov 04 07:42:21.228769 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:21.228737    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Nov 04 07:42:21.228937 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:21.228790    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" podUID="ff9f13c6-f606-4e5a-88dc-3987c2092418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Nov 04 07:42:21.323925 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:21.323881    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-df3be8/simulator-w0-df3be8-kserve-7d67d8c5b9-76j25" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 07:42:22.096023 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:22.095985    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" path="/var/lib/kubelet/pods/24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd/volumes"
Nov 04 07:42:22.096607 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:22.096590    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" path="/var/lib/kubelet/pods/cf8055f2-ea85-4b51-a7ce-766f09b12126/volumes"
Nov 04 07:42:22.232387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:22.230492    5626 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Nov 04 07:42:22.232387 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:22.230556    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" podUID="ff9f13c6-f606-4e5a-88dc-3987c2092418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Nov 04 07:42:23.469118 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:23.469077    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:23.469737 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:23.469707    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:23.470590 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:23.470569    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:24.831898 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:24.831821    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn" podUID="dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a" containerName="payload-processing" containerID="cri-o://f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3" gracePeriod=25
Nov 04 07:42:24.842912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:24.842883    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm" podUID="69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3" containerName="payload-pre-processing" containerID="cri-o://ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54" gracePeriod=25
Nov 04 07:42:25.099015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.098958    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"
Nov 04 07:42:25.128741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.128718    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"
Nov 04 07:42:25.222993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.222965    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-plugins-config-volume\") pod \"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3\" (UID: \"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3\") "
Nov 04 07:42:25.223152 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.223003    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-kube-api-access-79j96\" (UniqueName: \"kubernetes.io/projected/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-kube-api-access-79j96\") pod \"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3\" (UID: \"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3\") "
Nov 04 07:42:25.223152 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.223051    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-kube-api-access-wrp87\" (UniqueName: \"kubernetes.io/projected/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-kube-api-access-wrp87\") pod \"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a\" (UID: \"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a\") "
Nov 04 07:42:25.223152 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.223129    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-plugins-config-volume\") pod \"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a\" (UID: \"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a\") "
Nov 04 07:42:25.223438 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.223405    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-plugins-config-volume" pod "69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3" (UID: "69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:25.223635 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.223607    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-plugins-config-volume" pod "dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a" (UID: "dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:25.225204 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.225182    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-kube-api-access-wrp87" pod "dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a" (UID: "dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a"). InnerVolumeSpecName "kube-api-access-wrp87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:25.225278 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.225203    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-kube-api-access-79j96" pod "69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3" (UID: "69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3"). InnerVolumeSpecName "kube-api-access-79j96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:25.324153 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.324123    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:25.324153 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.324148    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:25.324153 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.324160    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79j96\" (UniqueName: \"kubernetes.io/projected/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3-kube-api-access-79j96\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:25.324392 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.324170    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrp87\" (UniqueName: \"kubernetes.io/projected/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a-kube-api-access-wrp87\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:25.841056 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.841017    5626 generic.go:356] "Generic (PLEG): container finished" podID="dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a" containerID="f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3" exitCode=0
Nov 04 07:42:25.841511 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.841059    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"
Nov 04 07:42:25.841511 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.841097    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn" event={"ID":"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a","Type":"ContainerDied","Data":"f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3"}
Nov 04 07:42:25.841511 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.841151    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn" event={"ID":"dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a","Type":"ContainerDied","Data":"543b7eab241ceafb3e7ab5e00d4d6e4467ef6b1be1f76d3e0aa1aaeebd48ece1"}
Nov 04 07:42:25.841511 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.841173    5626 scope.go:122] "RemoveContainer" containerID="f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3"
Nov 04 07:42:25.842912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.842885    5626 generic.go:356] "Generic (PLEG): container finished" podID="69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3" containerID="ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54" exitCode=0
Nov 04 07:42:25.843039 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.842942    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"
Nov 04 07:42:25.843039 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.842946    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm" event={"ID":"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3","Type":"ContainerDied","Data":"ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54"}
Nov 04 07:42:25.843039 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.843005    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm" event={"ID":"69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3","Type":"ContainerDied","Data":"fb37a507115eb63e81a8aa4ccb8115df827ac31f6b8a223b00bb682407153e27"}
Nov 04 07:42:25.864168 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.863994    5626 scope.go:122] "RemoveContainer" containerID="f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3"
Nov 04 07:42:25.864306 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:25.864285    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3\": container with ID starting with f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3 not found: ID does not exist" containerID="f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3"
Nov 04 07:42:25.864391 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.864315    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3"} err="failed to get container status \"f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3\": rpc error: code = NotFound desc = could not find container \"f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3\": container with ID starting with f4fb427c33c3770102e86fc40963010f58de6bed32aada80a0ba696d0e341ad3 not found: ID does not exist"
Nov 04 07:42:25.864391 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.864332    5626 scope.go:122] "RemoveContainer" containerID="ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54"
Nov 04 07:42:25.874201 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.874183    5626 scope.go:122] "RemoveContainer" containerID="ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54"
Nov 04 07:42:25.874547 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:25.874529    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54\": container with ID starting with ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54 not found: ID does not exist" containerID="ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54"
Nov 04 07:42:25.874605 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.874556    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54"} err="failed to get container status \"ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54\": rpc error: code = NotFound desc = could not find container \"ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54\": container with ID starting with ad897d25d21bb49bdf6440cc98a222782886429695f634776e632d54f88f7c54 not found: ID does not exist"
Nov 04 07:42:25.875632 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.875613    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"]
Nov 04 07:42:25.882830 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.882807    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm"]
Nov 04 07:42:25.894433 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.894412    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"]
Nov 04 07:42:25.898667 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:25.898641    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn"]
Nov 04 07:42:26.093274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:26.093196    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3" path="/var/lib/kubelet/pods/69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3/volumes"
Nov 04 07:42:26.093660 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:26.093642    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a" path="/var/lib/kubelet/pods/dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a/volumes"
Nov 04 07:42:26.244981 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:26.244945    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"]
Nov 04 07:42:26.245246 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:26.245219    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl" podUID="695aa56c-30fd-4351-abb6-b859413774e4" containerName="istio-proxy" containerID="cri-o://f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31" gracePeriod=30
Nov 04 07:42:26.248564 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:26.248540    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"]
Nov 04 07:42:26.248716 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:26.248697    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" podUID="ff9f13c6-f606-4e5a-88dc-3987c2092418" containerName="istio-proxy" containerID="cri-o://168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e" gracePeriod=30
Nov 04 07:42:26.254581 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:26.254554    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"]
Nov 04 07:42:26.254735 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:26.254716    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" podUID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" containerName="istio-proxy" containerID="cri-o://9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2" gracePeriod=30
Nov 04 07:42:26.504861 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:26.504775    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv" podUID="9661dd1c-40ae-420f-96cf-af9a01477333" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:42:26.505094 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:26.505071    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b" podUID="3477aa8d-a28d-4e96-8a17-1f177c1080ff" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:42:31.277158 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277127    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"]
Nov 04 07:42:31.277742 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277720    5626 cpu_manager.go:414] "containerMap: removing container" podUID="69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3" containerName="payload-pre-processing"
Nov 04 07:42:31.277742 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277742    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3" containerName="payload-pre-processing"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277757    5626 cpu_manager.go:414] "containerMap: removing container" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277763    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277771    5626 cpu_manager.go:414] "containerMap: removing container" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277777    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277790    5626 cpu_manager.go:414] "containerMap: removing container" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277795    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277814    5626 cpu_manager.go:414] "containerMap: removing container" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277819    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277825    5626 cpu_manager.go:414] "containerMap: removing container" podUID="dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a" containerName="payload-processing"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277829    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a" containerName="payload-processing"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277836    5626 cpu_manager.go:414] "containerMap: removing container" podUID="0e3d2c37-c22a-41d5-985b-6ab478e9528a" containerName="revoke-keys"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277840    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e3d2c37-c22a-41d5-985b-6ab478e9528a" containerName="revoke-keys"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277857    5626 cpu_manager.go:414] "containerMap: removing container" podUID="efa49e6f-5463-4724-a2b6-51882cb8b317" containerName="payload-pre-processing"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277864    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efa49e6f-5463-4724-a2b6-51882cb8b317" containerName="payload-pre-processing"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277871    5626 cpu_manager.go:414] "containerMap: removing container" podUID="7244c03f-1118-4bab-92ec-34894a8c9c5f" containerName="payload-processing"
Nov 04 07:42:31.277895 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277875    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7244c03f-1118-4bab-92ec-34894a8c9c5f" containerName="payload-processing"
Nov 04 07:42:31.278957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277955    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="69fe62cf-c6f0-4fc5-83fb-151e33cfa0c3" containerName="payload-pre-processing"
Nov 04 07:42:31.278957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277967    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy"
Nov 04 07:42:31.278957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277973    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="24be67c2-869c-4e5b-ad7c-8bf19b2ee6cd" containerName="main"
Nov 04 07:42:31.278957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277981    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="222aacaf-0c37-4aad-91ad-5607a50820ce" containerName="istio-proxy"
Nov 04 07:42:31.278957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277988    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf8055f2-ea85-4b51-a7ce-766f09b12126" containerName="main"
Nov 04 07:42:31.278957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.277994    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbc77902-2fd9-47b8-b8f3-b95a2ecd0b7a" containerName="payload-processing"
Nov 04 07:42:31.278957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.278000    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e3d2c37-c22a-41d5-985b-6ab478e9528a" containerName="revoke-keys"
Nov 04 07:42:31.278957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.278007    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="efa49e6f-5463-4724-a2b6-51882cb8b317" containerName="payload-pre-processing"
Nov 04 07:42:31.278957 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.278012    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="7244c03f-1118-4bab-92ec-34894a8c9c5f" containerName="payload-processing"
Nov 04 07:42:31.281222 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.281202    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:31.288239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.288206    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:42:31.289266 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.289229    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:42:31.289595 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.289575    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:42:31.289743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.289630    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:42:31.293857 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.293835    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"]
Nov 04 07:42:31.397310 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.397261    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhf5m\" (UniqueName: \"kubernetes.io/projected/e30c2240-1bc9-4538-8f44-8cf3c22c1552-kube-api-access-jhf5m\") pod \"maas-api-revoke-sub-1c7e5d0b-4c8qh\" (UID: \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:31.397512 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.397485    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1c7e5d0b-4c8qh\" (UID: \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:31.397576 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.397544    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1c7e5d0b-4c8qh\" (UID: \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:31.499605 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.499575    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1c7e5d0b-4c8qh\" (UID: \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:31.499759 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.499633    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1c7e5d0b-4c8qh\" (UID: \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:31.499918 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.499892    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhf5m\" (UniqueName: \"kubernetes.io/projected/e30c2240-1bc9-4538-8f44-8cf3c22c1552-kube-api-access-jhf5m\") pod \"maas-api-revoke-sub-1c7e5d0b-4c8qh\" (UID: \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:31.500598 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.500571    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1c7e5d0b-4c8qh\" (UID: \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:31.502757 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.502733    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1c7e5d0b-4c8qh\" (UID: \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:31.513264 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.513223    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhf5m\" (UniqueName: \"kubernetes.io/projected/e30c2240-1bc9-4538-8f44-8cf3c22c1552-kube-api-access-jhf5m\") pod \"maas-api-revoke-sub-1c7e5d0b-4c8qh\" (UID: \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:31.547622 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.547600    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:31.575979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.575950    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:42:31.594448 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.594427    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:42:31.701832 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.701800    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-kube-api-access-d2t8z\" (UniqueName: \"kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-kube-api-access-d2t8z\") pod \"695aa56c-30fd-4351-abb6-b859413774e4\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") "
Nov 04 07:42:31.702019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.701868    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-kube-api-access-xrwqx\" (UniqueName: \"kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-kube-api-access-xrwqx\") pod \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") "
Nov 04 07:42:31.702019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.701905    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-credential-socket\") pod \"ff9f13c6-f606-4e5a-88dc-3987c2092418\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") "
Nov 04 07:42:31.702019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.701956    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-data\") pod \"ff9f13c6-f606-4e5a-88dc-3987c2092418\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") "
Nov 04 07:42:31.702019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.701988    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/695aa56c-30fd-4351-abb6-b859413774e4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/695aa56c-30fd-4351-abb6-b859413774e4-istiod-ca-cert\") pod \"695aa56c-30fd-4351-abb6-b859413774e4\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") "
Nov 04 07:42:31.702236 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702019    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff9f13c6-f606-4e5a-88dc-3987c2092418-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff9f13c6-f606-4e5a-88dc-3987c2092418-istiod-ca-cert\") pod \"ff9f13c6-f606-4e5a-88dc-3987c2092418\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") "
Nov 04 07:42:31.702236 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702059    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-credential-socket\") pod \"695aa56c-30fd-4351-abb6-b859413774e4\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") "
Nov 04 07:42:31.702236 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702091    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-istio-token\" (UniqueName: \"kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-istio-token\") pod \"695aa56c-30fd-4351-abb6-b859413774e4\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") "
Nov 04 07:42:31.702236 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702123    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-envoy\") pod \"695aa56c-30fd-4351-abb6-b859413774e4\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") "
Nov 04 07:42:31.702236 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702173    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-envoy\") pod \"ff9f13c6-f606-4e5a-88dc-3987c2092418\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") "
Nov 04 07:42:31.702236 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702216    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-data\") pod \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") "
Nov 04 07:42:31.702579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702267    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-socket\") pod \"695aa56c-30fd-4351-abb6-b859413774e4\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") "
Nov 04 07:42:31.702579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702309    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-podinfo\") pod \"ff9f13c6-f606-4e5a-88dc-3987c2092418\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") "
Nov 04 07:42:31.702579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702343    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-envoy\") pod \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") "
Nov 04 07:42:31.702579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702396    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-socket\") pod \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") "
Nov 04 07:42:31.702579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702465    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-kube-api-access-rjbqc\" (UniqueName: \"kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-kube-api-access-rjbqc\") pod \"ff9f13c6-f606-4e5a-88dc-3987c2092418\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") "
Nov 04 07:42:31.702579 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702497    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff9f13c6-f606-4e5a-88dc-3987c2092418-istiod-ca-cert" pod "ff9f13c6-f606-4e5a-88dc-3987c2092418" (UID: "ff9f13c6-f606-4e5a-88dc-3987c2092418"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:31.702887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702584    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istiod-ca-cert\") pod \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") "
Nov 04 07:42:31.702887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702634    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-podinfo\") pod \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") "
Nov 04 07:42:31.702887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702667    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-token\" (UniqueName: \"kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-token\") pod \"ff9f13c6-f606-4e5a-88dc-3987c2092418\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") "
Nov 04 07:42:31.702887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702698    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-socket\") pod \"ff9f13c6-f606-4e5a-88dc-3987c2092418\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") "
Nov 04 07:42:31.702887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702729    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-data\") pod \"695aa56c-30fd-4351-abb6-b859413774e4\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") "
Nov 04 07:42:31.702887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702764    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/695aa56c-30fd-4351-abb6-b859413774e4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/695aa56c-30fd-4351-abb6-b859413774e4-istio-podinfo\") pod \"695aa56c-30fd-4351-abb6-b859413774e4\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") "
Nov 04 07:42:31.702887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702772    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-credential-socket" pod "ff9f13c6-f606-4e5a-88dc-3987c2092418" (UID: "ff9f13c6-f606-4e5a-88dc-3987c2092418"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.702887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702800    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-credential-socket\") pod \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") "
Nov 04 07:42:31.702887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702837    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-token\" (UniqueName: \"kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-token\") pod \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") "
Nov 04 07:42:31.702887 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702864    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-certs\") pod \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\" (UID: \"cfdec595-bd19-4d94-9a93-5bfc67f9de4e\") "
Nov 04 07:42:31.703234 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702904    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-certs\") pod \"ff9f13c6-f606-4e5a-88dc-3987c2092418\" (UID: \"ff9f13c6-f606-4e5a-88dc-3987c2092418\") "
Nov 04 07:42:31.703234 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.702934    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-certs\") pod \"695aa56c-30fd-4351-abb6-b859413774e4\" (UID: \"695aa56c-30fd-4351-abb6-b859413774e4\") "
Nov 04 07:42:31.703234 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.703012    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-data" pod "ff9f13c6-f606-4e5a-88dc-3987c2092418" (UID: "ff9f13c6-f606-4e5a-88dc-3987c2092418"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.703405 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.703290    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/695aa56c-30fd-4351-abb6-b859413774e4-istiod-ca-cert" pod "695aa56c-30fd-4351-abb6-b859413774e4" (UID: "695aa56c-30fd-4351-abb6-b859413774e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:31.703606 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.703580    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-certs" pod "695aa56c-30fd-4351-abb6-b859413774e4" (UID: "695aa56c-30fd-4351-abb6-b859413774e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.703688 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.703619    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-data" pod "695aa56c-30fd-4351-abb6-b859413774e4" (UID: "695aa56c-30fd-4351-abb6-b859413774e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.703751 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.703720    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istiod-ca-cert" pod "cfdec595-bd19-4d94-9a93-5bfc67f9de4e" (UID: "cfdec595-bd19-4d94-9a93-5bfc67f9de4e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:31.703936 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.703866    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-credential-socket" pod "695aa56c-30fd-4351-abb6-b859413774e4" (UID: "695aa56c-30fd-4351-abb6-b859413774e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.704089 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.703973    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-credential-socket" pod "cfdec595-bd19-4d94-9a93-5bfc67f9de4e" (UID: "cfdec595-bd19-4d94-9a93-5bfc67f9de4e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.704206 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.704026    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.704376 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.704347    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.704501 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.704483    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/695aa56c-30fd-4351-abb6-b859413774e4-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.704612 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.703984    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-certs" pod "cfdec595-bd19-4d94-9a93-5bfc67f9de4e" (UID: "cfdec595-bd19-4d94-9a93-5bfc67f9de4e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.704691 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.704153    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-certs" pod "ff9f13c6-f606-4e5a-88dc-3987c2092418" (UID: "ff9f13c6-f606-4e5a-88dc-3987c2092418"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.704691 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.704338    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-socket" pod "695aa56c-30fd-4351-abb6-b859413774e4" (UID: "695aa56c-30fd-4351-abb6-b859413774e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.704691 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.704465    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-socket" pod "ff9f13c6-f606-4e5a-88dc-3987c2092418" (UID: "ff9f13c6-f606-4e5a-88dc-3987c2092418"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.704852 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.704831    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-data" pod "cfdec595-bd19-4d94-9a93-5bfc67f9de4e" (UID: "cfdec595-bd19-4d94-9a93-5bfc67f9de4e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.704939 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.704918    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff9f13c6-f606-4e5a-88dc-3987c2092418-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.705108 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.705082    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-kube-api-access-d2t8z" pod "695aa56c-30fd-4351-abb6-b859413774e4" (UID: "695aa56c-30fd-4351-abb6-b859413774e4"). InnerVolumeSpecName "kube-api-access-d2t8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:31.705532 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.705457    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-kube-api-access-rjbqc" pod "ff9f13c6-f606-4e5a-88dc-3987c2092418" (UID: "ff9f13c6-f606-4e5a-88dc-3987c2092418"). InnerVolumeSpecName "kube-api-access-rjbqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:31.705635 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.705577    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-socket" pod "cfdec595-bd19-4d94-9a93-5bfc67f9de4e" (UID: "cfdec595-bd19-4d94-9a93-5bfc67f9de4e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.707751 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.707723    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-podinfo" pod "ff9f13c6-f606-4e5a-88dc-3987c2092418" (UID: "ff9f13c6-f606-4e5a-88dc-3987c2092418"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:42:31.707751 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.707739    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-envoy" pod "ff9f13c6-f606-4e5a-88dc-3987c2092418" (UID: "ff9f13c6-f606-4e5a-88dc-3987c2092418"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.708695 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.708671    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-istio-token" pod "695aa56c-30fd-4351-abb6-b859413774e4" (UID: "695aa56c-30fd-4351-abb6-b859413774e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:31.708695 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.708686    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-kube-api-access-xrwqx" pod "cfdec595-bd19-4d94-9a93-5bfc67f9de4e" (UID: "cfdec595-bd19-4d94-9a93-5bfc67f9de4e"). InnerVolumeSpecName "kube-api-access-xrwqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:31.708861 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.708695    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-token" pod "cfdec595-bd19-4d94-9a93-5bfc67f9de4e" (UID: "cfdec595-bd19-4d94-9a93-5bfc67f9de4e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:31.708861 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.708687    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-podinfo" pod "cfdec595-bd19-4d94-9a93-5bfc67f9de4e" (UID: "cfdec595-bd19-4d94-9a93-5bfc67f9de4e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:42:31.708861 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.708718    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-token" pod "ff9f13c6-f606-4e5a-88dc-3987c2092418" (UID: "ff9f13c6-f606-4e5a-88dc-3987c2092418"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:31.708861 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.708770    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-envoy" pod "cfdec595-bd19-4d94-9a93-5bfc67f9de4e" (UID: "cfdec595-bd19-4d94-9a93-5bfc67f9de4e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.709703 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.709677    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/695aa56c-30fd-4351-abb6-b859413774e4-istio-podinfo" pod "695aa56c-30fd-4351-abb6-b859413774e4" (UID: "695aa56c-30fd-4351-abb6-b859413774e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:42:31.710018 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.709993    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-envoy" pod "695aa56c-30fd-4351-abb6-b859413774e4" (UID: "695aa56c-30fd-4351-abb6-b859413774e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:31.744665 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.744642    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809250    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809289    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809304    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809318    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809332    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809346    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809377    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809391    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809405    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809419    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjbqc\" (UniqueName: \"kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-kube-api-access-rjbqc\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809436    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809452    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809465    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9f13c6-f606-4e5a-88dc-3987c2092418-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809479    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809492    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809505    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/695aa56c-30fd-4351-abb6-b859413774e4-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809519    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809533    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809548    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809561    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9f13c6-f606-4e5a-88dc-3987c2092418-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809575    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/695aa56c-30fd-4351-abb6-b859413774e4-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809588    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2t8z\" (UniqueName: \"kubernetes.io/projected/695aa56c-30fd-4351-abb6-b859413774e4-kube-api-access-d2t8z\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.809629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.809604    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrwqx\" (UniqueName: \"kubernetes.io/projected/cfdec595-bd19-4d94-9a93-5bfc67f9de4e-kube-api-access-xrwqx\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:31.885294 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.885260    5626 generic.go:356] "Generic (PLEG): container finished" podID="ff9f13c6-f606-4e5a-88dc-3987c2092418" containerID="168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e" exitCode=0
Nov 04 07:42:31.885483 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.885329    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"
Nov 04 07:42:31.885483 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.885346    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" event={"ID":"ff9f13c6-f606-4e5a-88dc-3987c2092418","Type":"ContainerDied","Data":"168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e"}
Nov 04 07:42:31.885483 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.885455    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd" event={"ID":"ff9f13c6-f606-4e5a-88dc-3987c2092418","Type":"ContainerDied","Data":"e3163fed070ea3a2a8e9543b9cf60ee9a84d11a0169c9c89fd357535c6309dd1"}
Nov 04 07:42:31.885586 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.885489    5626 scope.go:122] "RemoveContainer" containerID="168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e"
Nov 04 07:42:31.887027 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.887005    5626 generic.go:356] "Generic (PLEG): container finished" podID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" containerID="9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2" exitCode=0
Nov 04 07:42:31.887132 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.887050    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"
Nov 04 07:42:31.887132 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.887090    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" event={"ID":"cfdec595-bd19-4d94-9a93-5bfc67f9de4e","Type":"ContainerDied","Data":"9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2"}
Nov 04 07:42:31.887244 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.887141    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr" event={"ID":"cfdec595-bd19-4d94-9a93-5bfc67f9de4e","Type":"ContainerDied","Data":"90c7bd6a1e5f17612c2b0ede2f4d3d75c72efad3a4f7737d7fdc3fb3d832fced"}
Nov 04 07:42:31.888800 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.888779    5626 generic.go:356] "Generic (PLEG): container finished" podID="695aa56c-30fd-4351-abb6-b859413774e4" containerID="f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31" exitCode=0
Nov 04 07:42:31.888914 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.888826    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"
Nov 04 07:42:31.888914 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.888851    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl" event={"ID":"695aa56c-30fd-4351-abb6-b859413774e4","Type":"ContainerDied","Data":"f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31"}
Nov 04 07:42:31.888914 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.888908    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl" event={"ID":"695aa56c-30fd-4351-abb6-b859413774e4","Type":"ContainerDied","Data":"9525a327dbd9314ef0669c393f4e116eca32030fa66d1e89f15e06e7d5a0c5f0"}
Nov 04 07:42:31.898131 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.898115    5626 scope.go:122] "RemoveContainer" containerID="168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e"
Nov 04 07:42:31.898475 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:31.898456    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e\": container with ID starting with 168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e not found: ID does not exist" containerID="168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e"
Nov 04 07:42:31.898548 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.898483    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e"} err="failed to get container status \"168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e\": rpc error: code = NotFound desc = could not find container \"168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e\": container with ID starting with 168065af275bfa392d2354a0b3aa14464fd9d64564081f0f3ae1a620610cb79e not found: ID does not exist"
Nov 04 07:42:31.898548 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.898502    5626 scope.go:122] "RemoveContainer" containerID="9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2"
Nov 04 07:42:31.907930 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.907911    5626 scope.go:122] "RemoveContainer" containerID="9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2"
Nov 04 07:42:31.908212 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:31.908192    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2\": container with ID starting with 9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2 not found: ID does not exist" containerID="9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2"
Nov 04 07:42:31.908263 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.908223    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2"} err="failed to get container status \"9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2\": rpc error: code = NotFound desc = could not find container \"9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2\": container with ID starting with 9740886f6d5ea872bb9f73eb3b6b60b9d9fd737a4d61cd5587c99d21769102e2 not found: ID does not exist"
Nov 04 07:42:31.908263 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.908239    5626 scope.go:122] "RemoveContainer" containerID="f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31"
Nov 04 07:42:31.913528 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.913508    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"]
Nov 04 07:42:31.920054 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.920033    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd"]
Nov 04 07:42:31.920339 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.920324    5626 scope.go:122] "RemoveContainer" containerID="f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31"
Nov 04 07:42:31.920722 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:31.920703    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31\": container with ID starting with f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31 not found: ID does not exist" containerID="f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31"
Nov 04 07:42:31.920797 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.920728    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31"} err="failed to get container status \"f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31\": rpc error: code = NotFound desc = could not find container \"f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31\": container with ID starting with f7a5c44dcdac169b642a7a86b4b5de3d6856854e056cf6200bf9aaf3bbe0ab31 not found: ID does not exist"
Nov 04 07:42:31.929916 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.929896    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"]
Nov 04 07:42:31.938205 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.938186    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr"]
Nov 04 07:42:31.951735 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.951712    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"]
Nov 04 07:42:31.956296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:31.956276    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl"]
Nov 04 07:42:32.032271 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.032243    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"]
Nov 04 07:42:32.032396 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:42:32.032285    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode30c2240_1bc9_4538_8f44_8cf3c22c1552.slice/crio-bc4b34bdc6ce0166e952e645bf10bbfe8e9278a0aa6ac234c546b9c8176a2078 WatchSource:0}: Error finding container bc4b34bdc6ce0166e952e645bf10bbfe8e9278a0aa6ac234c546b9c8176a2078: Status 404 returned error can't find the container with id bc4b34bdc6ce0166e952e645bf10bbfe8e9278a0aa6ac234c546b9c8176a2078
Nov 04 07:42:32.070389 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.070330    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"]
Nov 04 07:42:32.071081 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.071059    5626 cpu_manager.go:414] "containerMap: removing container" podUID="ff9f13c6-f606-4e5a-88dc-3987c2092418" containerName="istio-proxy"
Nov 04 07:42:32.071178 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.071084    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff9f13c6-f606-4e5a-88dc-3987c2092418" containerName="istio-proxy"
Nov 04 07:42:32.071178 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.071114    5626 cpu_manager.go:414] "containerMap: removing container" podUID="695aa56c-30fd-4351-abb6-b859413774e4" containerName="istio-proxy"
Nov 04 07:42:32.071178 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.071123    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="695aa56c-30fd-4351-abb6-b859413774e4" containerName="istio-proxy"
Nov 04 07:42:32.071178 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.071168    5626 cpu_manager.go:414] "containerMap: removing container" podUID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" containerName="istio-proxy"
Nov 04 07:42:32.071178 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.071179    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" containerName="istio-proxy"
Nov 04 07:42:32.071461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.071300    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" containerName="istio-proxy"
Nov 04 07:42:32.071461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.071320    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff9f13c6-f606-4e5a-88dc-3987c2092418" containerName="istio-proxy"
Nov 04 07:42:32.071461 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.071334    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="695aa56c-30fd-4351-abb6-b859413774e4" containerName="istio-proxy"
Nov 04 07:42:32.076425 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.076404    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.113214 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.113176    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="695aa56c-30fd-4351-abb6-b859413774e4" path="/var/lib/kubelet/pods/695aa56c-30fd-4351-abb6-b859413774e4/volumes"
Nov 04 07:42:32.113760 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.113736    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfdec595-bd19-4d94-9a93-5bfc67f9de4e" path="/var/lib/kubelet/pods/cfdec595-bd19-4d94-9a93-5bfc67f9de4e/volumes"
Nov 04 07:42:32.114261 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.114237    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff9f13c6-f606-4e5a-88dc-3987c2092418" path="/var/lib/kubelet/pods/ff9f13c6-f606-4e5a-88dc-3987c2092418/volumes"
Nov 04 07:42:32.115113 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.115090    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"]
Nov 04 07:42:32.214055 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.214021    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.214055 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.214058    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.214286 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.214083    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.214286 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.214275    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.214410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.214306    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.214410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.214333    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crckc\" (UniqueName: \"kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-kube-api-access-crckc\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.214525 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.214505    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.214604 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.214588    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.214665 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.214651    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.315660 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.315570    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.315660 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.315621    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.315660 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.315638    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.316319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.315678    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.316319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.315706    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.316319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.315727    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crckc\" (UniqueName: \"kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-kube-api-access-crckc\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.316319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.315800    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.316319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.315884    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.316319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.315919    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.316319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.316118    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.316319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.316155    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.316319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.316269    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.316319 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.316311    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.316743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.316501    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.318063 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.318040    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.318440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.318418    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.328432 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.328409    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.328621 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.328600    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crckc\" (UniqueName: \"kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-kube-api-access-crckc\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.415454 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.415429    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:32.697926 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.697902    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"]
Nov 04 07:42:32.698660 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:42:32.698632    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2b86fa0_09d3_49d8_b4dd_d6c356d41cf6.slice/crio-d049ec7db751dae80173f86f6fd7e3319c84f6ca788ac60a983035875ce9a7e4 WatchSource:0}: Error finding container d049ec7db751dae80173f86f6fd7e3319c84f6ca788ac60a983035875ce9a7e4: Status 404 returned error can't find the container with id d049ec7db751dae80173f86f6fd7e3319c84f6ca788ac60a983035875ce9a7e4
Nov 04 07:42:32.700969 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.700935    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:42:32.701027 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.701003    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:42:32.701071 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.701051    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:42:32.896774 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.896741    5626 generic.go:356] "Generic (PLEG): container finished" podID="e30c2240-1bc9-4538-8f44-8cf3c22c1552" containerID="b391758ad9be6854f942ead825b58da382068771dc0fd713f3b0113bc3229f1f" exitCode=0
Nov 04 07:42:32.896941 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.896780    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh" event={"ID":"e30c2240-1bc9-4538-8f44-8cf3c22c1552","Type":"ContainerDied","Data":"b391758ad9be6854f942ead825b58da382068771dc0fd713f3b0113bc3229f1f"}
Nov 04 07:42:32.896941 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.896837    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh" event={"ID":"e30c2240-1bc9-4538-8f44-8cf3c22c1552","Type":"ContainerStarted","Data":"bc4b34bdc6ce0166e952e645bf10bbfe8e9278a0aa6ac234c546b9c8176a2078"}
Nov 04 07:42:32.902031 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.901998    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" event={"ID":"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6","Type":"ContainerStarted","Data":"42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9"}
Nov 04 07:42:32.902152 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:32.902037    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" event={"ID":"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6","Type":"ContainerStarted","Data":"d049ec7db751dae80173f86f6fd7e3319c84f6ca788ac60a983035875ce9a7e4"}
Nov 04 07:42:33.416988 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:33.416953    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:33.418170 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:33.418147    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 04 07:42:33.418290 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:33.418197    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" podUID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 04 07:42:34.093070 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.093039    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:34.114965 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.114915    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" podStartSLOduration=2.114900379 podStartE2EDuration="2.114900379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:42:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:42:32.940596513 +0000 UTC m=+1574.962811773" watchObservedRunningTime="2026-11-04 07:42:34.114900379 +0000 UTC m=+1576.137115662"
Nov 04 07:42:34.236510 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.236404    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e30c2240-1bc9-4538-8f44-8cf3c22c1552-kube-api-access-jhf5m\" (UniqueName: \"kubernetes.io/projected/e30c2240-1bc9-4538-8f44-8cf3c22c1552-kube-api-access-jhf5m\") pod \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\" (UID: \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\") "
Nov 04 07:42:34.236510 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.236461    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-xks-ca\") pod \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\" (UID: \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\") "
Nov 04 07:42:34.236510 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.236488    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-openshift-ca\") pod \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\" (UID: \"e30c2240-1bc9-4538-8f44-8cf3c22c1552\") "
Nov 04 07:42:34.237030 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.236999    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-openshift-ca" pod "e30c2240-1bc9-4538-8f44-8cf3c22c1552" (UID: "e30c2240-1bc9-4538-8f44-8cf3c22c1552"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:34.237130 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.237112    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:34.263683 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.263334    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e30c2240-1bc9-4538-8f44-8cf3c22c1552-kube-api-access-jhf5m" pod "e30c2240-1bc9-4538-8f44-8cf3c22c1552" (UID: "e30c2240-1bc9-4538-8f44-8cf3c22c1552"). InnerVolumeSpecName "kube-api-access-jhf5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:34.263958 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.263891    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-xks-ca" pod "e30c2240-1bc9-4538-8f44-8cf3c22c1552" (UID: "e30c2240-1bc9-4538-8f44-8cf3c22c1552"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:42:34.352269 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.352160    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhf5m\" (UniqueName: \"kubernetes.io/projected/e30c2240-1bc9-4538-8f44-8cf3c22c1552-kube-api-access-jhf5m\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:34.352269 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.352216    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e30c2240-1bc9-4538-8f44-8cf3c22c1552-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:34.415983 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.415946    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 04 07:42:34.416171 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.416019    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" podUID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 04 07:42:34.925757 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.924532    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh"
Nov 04 07:42:34.925757 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.925512    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1c7e5d0b-4c8qh" event={"ID":"e30c2240-1bc9-4538-8f44-8cf3c22c1552","Type":"ContainerDied","Data":"bc4b34bdc6ce0166e952e645bf10bbfe8e9278a0aa6ac234c546b9c8176a2078"}
Nov 04 07:42:34.925757 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:34.925580    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc4b34bdc6ce0166e952e645bf10bbfe8e9278a0aa6ac234c546b9c8176a2078"
Nov 04 07:42:35.416574 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:35.416541    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 04 07:42:35.416743 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:35.416608    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" podUID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 04 07:42:35.875385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:35.875165    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"]
Nov 04 07:42:35.878912 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:35.878879    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"]
Nov 04 07:42:36.187349 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:36.187267    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:42:36.416043 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:36.416006    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 04 07:42:36.416243 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:36.416067    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" podUID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 04 07:42:36.505641 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:36.505550    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv" podUID="9661dd1c-40ae-420f-96cf-af9a01477333" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:42:36.505979 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:36.505952    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b" podUID="3477aa8d-a28d-4e96-8a17-1f177c1080ff" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:42:37.337307 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.337281    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"
Nov 04 07:42:37.384689 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.384665    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"
Nov 04 07:42:37.406681 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.406207    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9661dd1c-40ae-420f-96cf-af9a01477333-kube-api-access-8kg4x\" (UniqueName: \"kubernetes.io/projected/9661dd1c-40ae-420f-96cf-af9a01477333-kube-api-access-8kg4x\") pod \"9661dd1c-40ae-420f-96cf-af9a01477333\" (UID: \"9661dd1c-40ae-420f-96cf-af9a01477333\") "
Nov 04 07:42:37.406681 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.406287    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3477aa8d-a28d-4e96-8a17-1f177c1080ff-kube-api-access-8thbh\" (UniqueName: \"kubernetes.io/projected/3477aa8d-a28d-4e96-8a17-1f177c1080ff-kube-api-access-8thbh\") pod \"3477aa8d-a28d-4e96-8a17-1f177c1080ff\" (UID: \"3477aa8d-a28d-4e96-8a17-1f177c1080ff\") "
Nov 04 07:42:37.406681 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.406330    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9661dd1c-40ae-420f-96cf-af9a01477333-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9661dd1c-40ae-420f-96cf-af9a01477333-plugins-config-volume\") pod \"9661dd1c-40ae-420f-96cf-af9a01477333\" (UID: \"9661dd1c-40ae-420f-96cf-af9a01477333\") "
Nov 04 07:42:37.406681 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.406386    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3477aa8d-a28d-4e96-8a17-1f177c1080ff-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3477aa8d-a28d-4e96-8a17-1f177c1080ff-plugins-config-volume\") pod \"3477aa8d-a28d-4e96-8a17-1f177c1080ff\" (UID: \"3477aa8d-a28d-4e96-8a17-1f177c1080ff\") "
Nov 04 07:42:37.407215 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.407183    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3477aa8d-a28d-4e96-8a17-1f177c1080ff-plugins-config-volume" pod "3477aa8d-a28d-4e96-8a17-1f177c1080ff" (UID: "3477aa8d-a28d-4e96-8a17-1f177c1080ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:37.407967 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.407940    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9661dd1c-40ae-420f-96cf-af9a01477333-plugins-config-volume" pod "9661dd1c-40ae-420f-96cf-af9a01477333" (UID: "9661dd1c-40ae-420f-96cf-af9a01477333"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:37.410216 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.410179    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3477aa8d-a28d-4e96-8a17-1f177c1080ff-kube-api-access-8thbh" pod "3477aa8d-a28d-4e96-8a17-1f177c1080ff" (UID: "3477aa8d-a28d-4e96-8a17-1f177c1080ff"). InnerVolumeSpecName "kube-api-access-8thbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:37.410488 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.410466    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9661dd1c-40ae-420f-96cf-af9a01477333-kube-api-access-8kg4x" pod "9661dd1c-40ae-420f-96cf-af9a01477333" (UID: "9661dd1c-40ae-420f-96cf-af9a01477333"). InnerVolumeSpecName "kube-api-access-8kg4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:37.415870 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.415833    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 04 07:42:37.415953 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.415893    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" podUID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 04 07:42:37.507443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.507414    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kg4x\" (UniqueName: \"kubernetes.io/projected/9661dd1c-40ae-420f-96cf-af9a01477333-kube-api-access-8kg4x\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:37.507443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.507438    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8thbh\" (UniqueName: \"kubernetes.io/projected/3477aa8d-a28d-4e96-8a17-1f177c1080ff-kube-api-access-8thbh\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:37.507443 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.507450    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9661dd1c-40ae-420f-96cf-af9a01477333-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:37.507670 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.507460    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3477aa8d-a28d-4e96-8a17-1f177c1080ff-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:37.967783 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.967745    5626 generic.go:356] "Generic (PLEG): container finished" podID="9661dd1c-40ae-420f-96cf-af9a01477333" containerID="bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62" exitCode=0
Nov 04 07:42:37.967993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.967796    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"
Nov 04 07:42:37.967993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.967829    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv" event={"ID":"9661dd1c-40ae-420f-96cf-af9a01477333","Type":"ContainerDied","Data":"bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62"}
Nov 04 07:42:37.967993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.967887    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv" event={"ID":"9661dd1c-40ae-420f-96cf-af9a01477333","Type":"ContainerDied","Data":"1742923402cce20ae78658b196fc1cced92580eb2f9c33772990d777a0e0c036"}
Nov 04 07:42:37.967993 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.967914    5626 scope.go:122] "RemoveContainer" containerID="bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62"
Nov 04 07:42:37.969440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.969419    5626 generic.go:356] "Generic (PLEG): container finished" podID="3477aa8d-a28d-4e96-8a17-1f177c1080ff" containerID="ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23" exitCode=0
Nov 04 07:42:37.969566 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.969461    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"
Nov 04 07:42:37.969566 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.969500    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b" event={"ID":"3477aa8d-a28d-4e96-8a17-1f177c1080ff","Type":"ContainerDied","Data":"ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23"}
Nov 04 07:42:37.969566 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.969551    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b" event={"ID":"3477aa8d-a28d-4e96-8a17-1f177c1080ff","Type":"ContainerDied","Data":"b76cd21176df16df5c7c43e3da100ec1239f3173d7da3d6718471fa883a815e9"}
Nov 04 07:42:37.979320 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.979252    5626 scope.go:122] "RemoveContainer" containerID="bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62"
Nov 04 07:42:37.979587 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:37.979566    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62\": container with ID starting with bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62 not found: ID does not exist" containerID="bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62"
Nov 04 07:42:37.979682 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.979592    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62"} err="failed to get container status \"bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62\": rpc error: code = NotFound desc = could not find container \"bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62\": container with ID starting with bf8f0737dfed6699d51c1da6d862930e2614bccae28b8b069a8eadfcfcaa7b62 not found: ID does not exist"
Nov 04 07:42:37.979682 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.979610    5626 scope.go:122] "RemoveContainer" containerID="ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23"
Nov 04 07:42:37.990505 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.990460    5626 scope.go:122] "RemoveContainer" containerID="ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23"
Nov 04 07:42:37.990818 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:37.990797    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23\": container with ID starting with ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23 not found: ID does not exist" containerID="ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23"
Nov 04 07:42:37.990921 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.990825    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23"} err="failed to get container status \"ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23\": rpc error: code = NotFound desc = could not find container \"ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23\": container with ID starting with ffe8584eca3e3e0b5f5a5ce81170cca59f0f3467cc984b4b506440df8ff89e23 not found: ID does not exist"
Nov 04 07:42:37.996320 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:37.996297    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"]
Nov 04 07:42:38.004433 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:38.004400    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv"]
Nov 04 07:42:38.021535 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:38.021507    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"]
Nov 04 07:42:38.025530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:38.025503    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b"]
Nov 04 07:42:38.092499 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:38.092467    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3477aa8d-a28d-4e96-8a17-1f177c1080ff" path="/var/lib/kubelet/pods/3477aa8d-a28d-4e96-8a17-1f177c1080ff/volumes"
Nov 04 07:42:38.092985 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:38.092963    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9661dd1c-40ae-420f-96cf-af9a01477333" path="/var/lib/kubelet/pods/9661dd1c-40ae-420f-96cf-af9a01477333/volumes"
Nov 04 07:42:38.416156 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:38.416118    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 04 07:42:38.416621 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:38.416176    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" podUID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 04 07:42:39.416045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:39.416001    5626 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Nov 04 07:42:39.416210 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:39.416076    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" podUID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Nov 04 07:42:40.741252 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:40.741216    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:40.741748 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:40.741613    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:40.742844 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:40.742815    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:42.433757 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.433719    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"]
Nov 04 07:42:42.458937 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.458905    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"]
Nov 04 07:42:42.691489 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.691406    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"]
Nov 04 07:42:42.691884 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.691871    5626 cpu_manager.go:414] "containerMap: removing container" podUID="e30c2240-1bc9-4538-8f44-8cf3c22c1552" containerName="revoke-keys"
Nov 04 07:42:42.691949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.691885    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e30c2240-1bc9-4538-8f44-8cf3c22c1552" containerName="revoke-keys"
Nov 04 07:42:42.691949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.691913    5626 cpu_manager.go:414] "containerMap: removing container" podUID="9661dd1c-40ae-420f-96cf-af9a01477333" containerName="payload-pre-processing"
Nov 04 07:42:42.691949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.691919    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9661dd1c-40ae-420f-96cf-af9a01477333" containerName="payload-pre-processing"
Nov 04 07:42:42.691949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.691938    5626 cpu_manager.go:414] "containerMap: removing container" podUID="3477aa8d-a28d-4e96-8a17-1f177c1080ff" containerName="payload-processing"
Nov 04 07:42:42.691949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.691942    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3477aa8d-a28d-4e96-8a17-1f177c1080ff" containerName="payload-processing"
Nov 04 07:42:42.692096 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.692020    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="e30c2240-1bc9-4538-8f44-8cf3c22c1552" containerName="revoke-keys"
Nov 04 07:42:42.692096 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.692029    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="9661dd1c-40ae-420f-96cf-af9a01477333" containerName="payload-pre-processing"
Nov 04 07:42:42.692096 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.692039    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="3477aa8d-a28d-4e96-8a17-1f177c1080ff" containerName="payload-processing"
Nov 04 07:42:42.695431 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.695409    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:42.700735 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.700327    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 07:42:42.700735 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.700424    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 07:42:42.700735 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.700439    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxs5x\""
Nov 04 07:42:42.700735 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.700461    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 07:42:42.706515 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.706488    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"]
Nov 04 07:42:42.767274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.767242    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ca1547e-msm67\" (UID: \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:42.767470 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.767342    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pts7\" (UniqueName: \"kubernetes.io/projected/f8f994c3-eace-4fe2-8094-ccfbf67f6684-kube-api-access-7pts7\") pod \"maas-api-revoke-sub-9ca1547e-msm67\" (UID: \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:42.767470 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.767445    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ca1547e-msm67\" (UID: \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:42.868734 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.868699    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pts7\" (UniqueName: \"kubernetes.io/projected/f8f994c3-eace-4fe2-8094-ccfbf67f6684-kube-api-access-7pts7\") pod \"maas-api-revoke-sub-9ca1547e-msm67\" (UID: \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:42.868886 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.868793    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ca1547e-msm67\" (UID: \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:42.868886 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.868831    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ca1547e-msm67\" (UID: \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:42.869499 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.869472    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ca1547e-msm67\" (UID: \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:42.871205 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.871184    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ca1547e-msm67\" (UID: \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:42.883085 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:42.883058    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pts7\" (UniqueName: \"kubernetes.io/projected/f8f994c3-eace-4fe2-8094-ccfbf67f6684-kube-api-access-7pts7\") pod \"maas-api-revoke-sub-9ca1547e-msm67\" (UID: \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:43.014878 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:43.014799    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:43.291569 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:43.291534    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"]
Nov 04 07:42:43.292324 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:42:43.292299    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f994c3_eace_4fe2_8094_ccfbf67f6684.slice/crio-df59a8d3a58b3ea6b3254d42a80bc20c68451aa950899054b6794c2f8dc48336 WatchSource:0}: Error finding container df59a8d3a58b3ea6b3254d42a80bc20c68451aa950899054b6794c2f8dc48336: Status 404 returned error can't find the container with id df59a8d3a58b3ea6b3254d42a80bc20c68451aa950899054b6794c2f8dc48336
Nov 04 07:42:44.015348 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:44.015257    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67" event={"ID":"f8f994c3-eace-4fe2-8094-ccfbf67f6684","Type":"ContainerStarted","Data":"3da06ea6613eb9301ef4a2545006c7b1d1c5bc49eb7d1a0fa7778a1111fb164f"}
Nov 04 07:42:44.015348 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:44.015322    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67" event={"ID":"f8f994c3-eace-4fe2-8094-ccfbf67f6684","Type":"ContainerStarted","Data":"df59a8d3a58b3ea6b3254d42a80bc20c68451aa950899054b6794c2f8dc48336"}
Nov 04 07:42:44.036274 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:44.036224    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67" podStartSLOduration=2.036208541 podStartE2EDuration="2.036208541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:42:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:42:44.034073312 +0000 UTC m=+1586.056288573" watchObservedRunningTime="2026-11-04 07:42:44.036208541 +0000 UTC m=+1586.058423856"
Nov 04 07:42:44.577296 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:44.577260    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"]
Nov 04 07:42:44.577688 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:44.577535    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz" podUID="646c74b6-7721-410a-bccf-004404e9c41a" containerName="istio-proxy" containerID="cri-o://5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857" gracePeriod=30
Nov 04 07:42:44.591924 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:44.591870    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"]
Nov 04 07:42:44.592284 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:44.592258    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerName="istio-proxy" containerID="cri-o://fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44" gracePeriod=30
Nov 04 07:42:44.603984 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:44.603960    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"]
Nov 04 07:42:44.604143 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:44.604126    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" podUID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" containerName="istio-proxy" containerID="cri-o://42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9" gracePeriod=30
Nov 04 07:42:45.023109 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:45.023021    5626 generic.go:356] "Generic (PLEG): container finished" podID="f8f994c3-eace-4fe2-8094-ccfbf67f6684" containerID="3da06ea6613eb9301ef4a2545006c7b1d1c5bc49eb7d1a0fa7778a1111fb164f" exitCode=0
Nov 04 07:42:45.023109 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:45.023073    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67" event={"ID":"f8f994c3-eace-4fe2-8094-ccfbf67f6684","Type":"ContainerDied","Data":"3da06ea6613eb9301ef4a2545006c7b1d1c5bc49eb7d1a0fa7778a1111fb164f"}
Nov 04 07:42:46.181609 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:46.181576    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:46.317435 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:46.317330    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8f994c3-eace-4fe2-8094-ccfbf67f6684-kube-api-access-7pts7\" (UniqueName: \"kubernetes.io/projected/f8f994c3-eace-4fe2-8094-ccfbf67f6684-kube-api-access-7pts7\") pod \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\" (UID: \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\") "
Nov 04 07:42:46.317610 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:46.317451    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-xks-ca\") pod \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\" (UID: \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\") "
Nov 04 07:42:46.317610 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:46.317559    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-openshift-ca\") pod \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\" (UID: \"f8f994c3-eace-4fe2-8094-ccfbf67f6684\") "
Nov 04 07:42:46.317880 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:46.317850    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-openshift-ca" pod "f8f994c3-eace-4fe2-8094-ccfbf67f6684" (UID: "f8f994c3-eace-4fe2-8094-ccfbf67f6684"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:46.319535 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:46.319509    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-xks-ca" pod "f8f994c3-eace-4fe2-8094-ccfbf67f6684" (UID: "f8f994c3-eace-4fe2-8094-ccfbf67f6684"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:42:46.319650 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:46.319596    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8f994c3-eace-4fe2-8094-ccfbf67f6684-kube-api-access-7pts7" pod "f8f994c3-eace-4fe2-8094-ccfbf67f6684" (UID: "f8f994c3-eace-4fe2-8094-ccfbf67f6684"). InnerVolumeSpecName "kube-api-access-7pts7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:46.419129 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:46.419094    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-xks-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:46.419129 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:46.419123    5626 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f8f994c3-eace-4fe2-8094-ccfbf67f6684-maas-api-openshift-ca\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:46.419129 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:46.419133    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pts7\" (UniqueName: \"kubernetes.io/projected/f8f994c3-eace-4fe2-8094-ccfbf67f6684-kube-api-access-7pts7\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:47.038215 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:47.038185    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67"
Nov 04 07:42:47.038413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:47.038228    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ca1547e-msm67" event={"ID":"f8f994c3-eace-4fe2-8094-ccfbf67f6684","Type":"ContainerDied","Data":"df59a8d3a58b3ea6b3254d42a80bc20c68451aa950899054b6794c2f8dc48336"}
Nov 04 07:42:47.038413 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:47.038298    5626 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df59a8d3a58b3ea6b3254d42a80bc20c68451aa950899054b6794c2f8dc48336"
Nov 04 07:42:47.434053 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:47.433967    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn" podUID="41718dd1-af1a-4e5b-953a-f9734b3f3483" containerName="payload-processing" containerID="cri-o://c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012" gracePeriod=25
Nov 04 07:42:47.459573 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:47.459541    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm" podUID="c5e98a2b-fc18-4ec7-9fc0-34330c4fc120" containerName="payload-pre-processing" containerID="cri-o://4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829" gracePeriod=25
Nov 04 07:42:47.696328 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:47.696267    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:42:47.834584 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:47.834552    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41718dd1-af1a-4e5b-953a-f9734b3f3483-kube-api-access-xmxtm\" (UniqueName: \"kubernetes.io/projected/41718dd1-af1a-4e5b-953a-f9734b3f3483-kube-api-access-xmxtm\") pod \"41718dd1-af1a-4e5b-953a-f9734b3f3483\" (UID: \"41718dd1-af1a-4e5b-953a-f9734b3f3483\") "
Nov 04 07:42:47.834770 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:47.834617    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41718dd1-af1a-4e5b-953a-f9734b3f3483-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41718dd1-af1a-4e5b-953a-f9734b3f3483-plugins-config-volume\") pod \"41718dd1-af1a-4e5b-953a-f9734b3f3483\" (UID: \"41718dd1-af1a-4e5b-953a-f9734b3f3483\") "
Nov 04 07:42:47.835082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:47.835050    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41718dd1-af1a-4e5b-953a-f9734b3f3483-plugins-config-volume" pod "41718dd1-af1a-4e5b-953a-f9734b3f3483" (UID: "41718dd1-af1a-4e5b-953a-f9734b3f3483"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:47.836764 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:47.836738    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41718dd1-af1a-4e5b-953a-f9734b3f3483-kube-api-access-xmxtm" pod "41718dd1-af1a-4e5b-953a-f9734b3f3483" (UID: "41718dd1-af1a-4e5b-953a-f9734b3f3483"). InnerVolumeSpecName "kube-api-access-xmxtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:47.935556 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:47.935525    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmxtm\" (UniqueName: \"kubernetes.io/projected/41718dd1-af1a-4e5b-953a-f9734b3f3483-kube-api-access-xmxtm\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:47.935556 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:47.935550    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41718dd1-af1a-4e5b-953a-f9734b3f3483-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:48.046130 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:48.046097    5626 generic.go:356] "Generic (PLEG): container finished" podID="41718dd1-af1a-4e5b-953a-f9734b3f3483" containerID="c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012" exitCode=0
Nov 04 07:42:48.046272 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:48.046139    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"
Nov 04 07:42:48.046272 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:48.046142    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn" event={"ID":"41718dd1-af1a-4e5b-953a-f9734b3f3483","Type":"ContainerDied","Data":"c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012"}
Nov 04 07:42:48.046445 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:48.046267    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn" event={"ID":"41718dd1-af1a-4e5b-953a-f9734b3f3483","Type":"ContainerDied","Data":"6d44aede2ddd89987e078f5bd8461bf475bfe490c05d0105656260df81718156"}
Nov 04 07:42:48.046445 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:48.046300    5626 scope.go:122] "RemoveContainer" containerID="c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012"
Nov 04 07:42:48.058956 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:48.058935    5626 scope.go:122] "RemoveContainer" containerID="c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012"
Nov 04 07:42:48.059263 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:48.059244    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012\": container with ID starting with c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012 not found: ID does not exist" containerID="c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012"
Nov 04 07:42:48.059338 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:48.059272    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012"} err="failed to get container status \"c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012\": rpc error: code = NotFound desc = could not find container \"c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012\": container with ID starting with c742a2774345318b0f596e69ef914c2d2dbf360758247bf2dfcbf8bfce3f9012 not found: ID does not exist"
Nov 04 07:42:48.073532 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:48.073500    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"]
Nov 04 07:42:48.077027 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:48.077006    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn"]
Nov 04 07:42:48.091948 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:48.091907    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41718dd1-af1a-4e5b-953a-f9734b3f3483" path="/var/lib/kubelet/pods/41718dd1-af1a-4e5b-953a-f9734b3f3483/volumes"
Nov 04 07:42:49.867052 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.867026    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:42:49.889496 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.889461    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:42:49.917512 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.917488    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:49.954788 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.954758    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-podinfo\") pod \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") "
Nov 04 07:42:49.954916 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.954797    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-data\") pod \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") "
Nov 04 07:42:49.954916 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.954827    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-kube-api-access-rdtgv\" (UniqueName: \"kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-kube-api-access-rdtgv\") pod \"646c74b6-7721-410a-bccf-004404e9c41a\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") "
Nov 04 07:42:49.955027 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955004    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-credential-socket\") pod \"646c74b6-7721-410a-bccf-004404e9c41a\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") "
Nov 04 07:42:49.955068 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955053    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-certs\") pod \"646c74b6-7721-410a-bccf-004404e9c41a\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") "
Nov 04 07:42:49.955111 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955093    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/646c74b6-7721-410a-bccf-004404e9c41a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/646c74b6-7721-410a-bccf-004404e9c41a-istiod-ca-cert\") pod \"646c74b6-7721-410a-bccf-004404e9c41a\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") "
Nov 04 07:42:49.955154 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955107    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-data" pod "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" (UID: "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:49.955154 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955122    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-data\") pod \"646c74b6-7721-410a-bccf-004404e9c41a\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") "
Nov 04 07:42:49.955154 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955149    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-envoy\") pod \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") "
Nov 04 07:42:49.955302 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955188    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-token\" (UniqueName: \"kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-token\") pod \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") "
Nov 04 07:42:49.955302 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955256    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istiod-ca-cert\") pod \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") "
Nov 04 07:42:49.955439 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955304    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-kube-api-access-m6nhg\" (UniqueName: \"kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-kube-api-access-m6nhg\") pod \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") "
Nov 04 07:42:49.955439 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955339    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-envoy\") pod \"646c74b6-7721-410a-bccf-004404e9c41a\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") "
Nov 04 07:42:49.955439 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955374    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-credential-socket" pod "646c74b6-7721-410a-bccf-004404e9c41a" (UID: "646c74b6-7721-410a-bccf-004404e9c41a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:49.955439 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955410    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-socket\") pod \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") "
Nov 04 07:42:49.955655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955474    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-istio-token\" (UniqueName: \"kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-istio-token\") pod \"646c74b6-7721-410a-bccf-004404e9c41a\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") "
Nov 04 07:42:49.955655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955541    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-socket\") pod \"646c74b6-7721-410a-bccf-004404e9c41a\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") "
Nov 04 07:42:49.955655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955575    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-credential-socket\") pod \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") "
Nov 04 07:42:49.955655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955595    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-data" pod "646c74b6-7721-410a-bccf-004404e9c41a" (UID: "646c74b6-7721-410a-bccf-004404e9c41a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:49.955655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955609    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-certs\") pod \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\" (UID: \"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa\") "
Nov 04 07:42:49.955655 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955642    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/646c74b6-7721-410a-bccf-004404e9c41a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/646c74b6-7721-410a-bccf-004404e9c41a-istio-podinfo\") pod \"646c74b6-7721-410a-bccf-004404e9c41a\" (UID: \"646c74b6-7721-410a-bccf-004404e9c41a\") "
Nov 04 07:42:49.955969 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955747    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-certs" pod "646c74b6-7721-410a-bccf-004404e9c41a" (UID: "646c74b6-7721-410a-bccf-004404e9c41a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:49.955969 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955799    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-socket" pod "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" (UID: "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:49.955969 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.955877    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/646c74b6-7721-410a-bccf-004404e9c41a-istiod-ca-cert" pod "646c74b6-7721-410a-bccf-004404e9c41a" (UID: "646c74b6-7721-410a-bccf-004404e9c41a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:49.956133 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.956042    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-socket" pod "646c74b6-7721-410a-bccf-004404e9c41a" (UID: "646c74b6-7721-410a-bccf-004404e9c41a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:49.956240 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.956219    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-credential-socket" pod "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" (UID: "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:49.956688 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.956347    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-certs" pod "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" (UID: "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:49.956688 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.956589    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istiod-ca-cert" pod "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" (UID: "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:49.957380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.957215    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:49.957380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.957240    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:49.957380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.957255    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:49.957380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.957271    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:49.957380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.957285    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:49.957380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.957296    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:49.957380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.957309    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/646c74b6-7721-410a-bccf-004404e9c41a-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:49.957380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.957321    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:49.957380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.957335    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:49.957380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.957348    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:49.957950 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.957905    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-podinfo" pod "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" (UID: "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:42:49.958030 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.957955    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-kube-api-access-rdtgv" pod "646c74b6-7721-410a-bccf-004404e9c41a" (UID: "646c74b6-7721-410a-bccf-004404e9c41a"). InnerVolumeSpecName "kube-api-access-rdtgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:49.958404 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.958344    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-envoy" pod "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" (UID: "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:49.958404 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.958387    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-token" pod "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" (UID: "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:49.959143 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.959122    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-istio-token" pod "646c74b6-7721-410a-bccf-004404e9c41a" (UID: "646c74b6-7721-410a-bccf-004404e9c41a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:49.959933 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.959894    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/646c74b6-7721-410a-bccf-004404e9c41a-istio-podinfo" pod "646c74b6-7721-410a-bccf-004404e9c41a" (UID: "646c74b6-7721-410a-bccf-004404e9c41a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:42:49.960015 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.960003    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-envoy" pod "646c74b6-7721-410a-bccf-004404e9c41a" (UID: "646c74b6-7721-410a-bccf-004404e9c41a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:49.960082 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:49.960064    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-kube-api-access-m6nhg" pod "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" (UID: "eb5f0d1c-bf2b-4f0d-8556-a85713f541aa"). InnerVolumeSpecName "kube-api-access-m6nhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:50.057995 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.057966    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-token\" (UniqueName: \"kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-token\") pod \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") "
Nov 04 07:42:50.058177 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058022    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-socket\") pod \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") "
Nov 04 07:42:50.058177 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058053    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istiod-ca-cert\") pod \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") "
Nov 04 07:42:50.058177 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058094    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-credential-socket\") pod \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") "
Nov 04 07:42:50.058177 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058130    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-envoy\") pod \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") "
Nov 04 07:42:50.058419 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058190    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-data\") pod \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") "
Nov 04 07:42:50.058419 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058227    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-podinfo\") pod \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") "
Nov 04 07:42:50.058419 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058268    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-certs\") pod \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") "
Nov 04 07:42:50.058419 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058296    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-kube-api-access-crckc\" (UniqueName: \"kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-kube-api-access-crckc\") pod \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\" (UID: \"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6\") "
Nov 04 07:42:50.058419 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058404    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-socket" pod "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" (UID: "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:50.058687 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058623    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-credential-socket" pod "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" (UID: "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:50.058687 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058645    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.058850 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058684    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/646c74b6-7721-410a-bccf-004404e9c41a-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.058850 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058703    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.058850 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058700    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-certs" pod "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" (UID: "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:50.058850 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058718    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdtgv\" (UniqueName: \"kubernetes.io/projected/646c74b6-7721-410a-bccf-004404e9c41a-kube-api-access-rdtgv\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.058850 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058737    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.058850 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058751    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.058850 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058765    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.058850 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058779    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6nhg\" (UniqueName: \"kubernetes.io/projected/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa-kube-api-access-m6nhg\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.058850 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.058794    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/646c74b6-7721-410a-bccf-004404e9c41a-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.059284 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.059260    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istiod-ca-cert" pod "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" (UID: "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:42:50.059380 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.059342    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-data" pod "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" (UID: "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:50.061195 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.061174    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-token" pod "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" (UID: "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:50.061271 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.061173    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-podinfo" pod "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" (UID: "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:42:50.061271 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.061204    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-kube-api-access-crckc" pod "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" (UID: "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6"). InnerVolumeSpecName "kube-api-access-crckc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:50.061330 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.061263    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-envoy" pod "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" (UID: "a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:50.073563 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.073536    5626 generic.go:356] "Generic (PLEG): container finished" podID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" containerID="42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9" exitCode=0
Nov 04 07:42:50.073689 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.073614    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"
Nov 04 07:42:50.073689 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.073622    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" event={"ID":"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6","Type":"ContainerDied","Data":"42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9"}
Nov 04 07:42:50.073809 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.073722    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr" event={"ID":"a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6","Type":"ContainerDied","Data":"d049ec7db751dae80173f86f6fd7e3319c84f6ca788ac60a983035875ce9a7e4"}
Nov 04 07:42:50.073809 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.073756    5626 scope.go:122] "RemoveContainer" containerID="42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9"
Nov 04 07:42:50.074960 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.074934    5626 generic.go:356] "Generic (PLEG): container finished" podID="646c74b6-7721-410a-bccf-004404e9c41a" containerID="5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857" exitCode=0
Nov 04 07:42:50.075075 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.074993    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"
Nov 04 07:42:50.075410 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.075353    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz" event={"ID":"646c74b6-7721-410a-bccf-004404e9c41a","Type":"ContainerDied","Data":"5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857"}
Nov 04 07:42:50.075498 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.075442    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz" event={"ID":"646c74b6-7721-410a-bccf-004404e9c41a","Type":"ContainerDied","Data":"443a626ced16a2264e0ad0d3c3388540340854784457475a6849afca061ebe23"}
Nov 04 07:42:50.076644 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.076621    5626 generic.go:356] "Generic (PLEG): container finished" podID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerID="fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44" exitCode=0
Nov 04 07:42:50.076752 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.076647    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" event={"ID":"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa","Type":"ContainerDied","Data":"fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44"}
Nov 04 07:42:50.076752 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.076684    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5" event={"ID":"eb5f0d1c-bf2b-4f0d-8556-a85713f541aa","Type":"ContainerDied","Data":"eee25e3cd0b8c80f7b2b909c2256d6e74a28c009b080916d412568a59117b698"}
Nov 04 07:42:50.076752 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.076715    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"
Nov 04 07:42:50.084248 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.084228    5626 scope.go:122] "RemoveContainer" containerID="42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9"
Nov 04 07:42:50.084569 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:50.084550    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9\": container with ID starting with 42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9 not found: ID does not exist" containerID="42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9"
Nov 04 07:42:50.084636 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.084577    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9"} err="failed to get container status \"42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9\": rpc error: code = NotFound desc = could not find container \"42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9\": container with ID starting with 42e897657f6e00029ee064efc5b706c68ba95d680f3d284e7c0a8071f2a7edc9 not found: ID does not exist"
Nov 04 07:42:50.084636 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.084593    5626 scope.go:122] "RemoveContainer" containerID="5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857"
Nov 04 07:42:50.095021 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.095002    5626 scope.go:122] "RemoveContainer" containerID="5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857"
Nov 04 07:42:50.095411 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:50.095384    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857\": container with ID starting with 5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857 not found: ID does not exist" containerID="5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857"
Nov 04 07:42:50.095504 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.095418    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857"} err="failed to get container status \"5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857\": rpc error: code = NotFound desc = could not find container \"5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857\": container with ID starting with 5f7a7991726f76e1d0a05471e747954ab40d0ca5155a82460fbe013367fd3857 not found: ID does not exist"
Nov 04 07:42:50.095504 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.095449    5626 scope.go:122] "RemoveContainer" containerID="fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44"
Nov 04 07:42:50.104748 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.104730    5626 scope.go:122] "RemoveContainer" containerID="fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44"
Nov 04 07:42:50.105027 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:42:50.105007    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44\": container with ID starting with fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44 not found: ID does not exist" containerID="fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44"
Nov 04 07:42:50.105092 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.105034    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44"} err="failed to get container status \"fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44\": rpc error: code = NotFound desc = could not find container \"fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44\": container with ID starting with fa192a34bbd62fcd7b3576c6641cb1b38932a5fadb2fb442535dd6835cfdcc44 not found: ID does not exist"
Nov 04 07:42:50.110741 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.110720    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"]
Nov 04 07:42:50.115091 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.115070    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz"]
Nov 04 07:42:50.127016 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.126993    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"]
Nov 04 07:42:50.132370 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.132340    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr"]
Nov 04 07:42:50.144779 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.144758    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"]
Nov 04 07:42:50.149882 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.149860    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5"]
Nov 04 07:42:50.160172 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.160151    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.160297 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.160176    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crckc\" (UniqueName: \"kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-kube-api-access-crckc\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.160297 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.160196    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.160297 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.160210    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.160297 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.160224    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.160297 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.160238    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.160297 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.160252    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.160297 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.160267    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:50.897291 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.897201    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main" containerID="cri-o://0702c6a1e941f4aa7bea140ebe7d8714eceb186904117e5c9b4fe6c1739f4a9b" gracePeriod=45
Nov 04 07:42:50.900045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:50.900010    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main" containerID="cri-o://a795dc778676f539d477b36173eb1d46bae773433bce65c70ca87771d467acf3" gracePeriod=45
Nov 04 07:42:51.090316 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.090285    5626 generic.go:356] "Generic (PLEG): container finished" podID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerID="a795dc778676f539d477b36173eb1d46bae773433bce65c70ca87771d467acf3" exitCode=0
Nov 04 07:42:51.090560 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.090349    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" event={"ID":"b915520d-c11d-402e-b0ff-c01dfd5cdc4f","Type":"ContainerDied","Data":"a795dc778676f539d477b36173eb1d46bae773433bce65c70ca87771d467acf3"}
Nov 04 07:42:51.094454 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.094431    5626 generic.go:356] "Generic (PLEG): container finished" podID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerID="0702c6a1e941f4aa7bea140ebe7d8714eceb186904117e5c9b4fe6c1739f4a9b" exitCode=0
Nov 04 07:42:51.094600 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.094475    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" event={"ID":"6629b726-b52a-4100-9bfd-ac8f4faca253","Type":"ContainerDied","Data":"0702c6a1e941f4aa7bea140ebe7d8714eceb186904117e5c9b4fe6c1739f4a9b"}
Nov 04 07:42:51.172906 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.172883    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:42:51.175987 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.175967    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:42:51.272926 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.272897    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-dshm\") pod \"6629b726-b52a-4100-9bfd-ac8f4faca253\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") "
Nov 04 07:42:51.273112 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.272953    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-kube-api-access-74s2f\" (UniqueName: \"kubernetes.io/projected/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-kube-api-access-74s2f\") pod \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") "
Nov 04 07:42:51.273112 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.272987    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tls-certs\" (UniqueName: \"kubernetes.io/secret/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tls-certs\") pod \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") "
Nov 04 07:42:51.273112 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273022    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-home\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-home\") pod \"6629b726-b52a-4100-9bfd-ac8f4faca253\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") "
Nov 04 07:42:51.273112 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273051    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-model-cache\") pod \"6629b726-b52a-4100-9bfd-ac8f4faca253\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") "
Nov 04 07:42:51.273334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273145    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-tmp-dir\") pod \"6629b726-b52a-4100-9bfd-ac8f4faca253\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") "
Nov 04 07:42:51.273334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273196    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-dshm\") pod \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") "
Nov 04 07:42:51.273334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273228    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-model-cache\") pod \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") "
Nov 04 07:42:51.273334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273270    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6629b726-b52a-4100-9bfd-ac8f4faca253-kube-api-access-pt28t\" (UniqueName: \"kubernetes.io/projected/6629b726-b52a-4100-9bfd-ac8f4faca253-kube-api-access-pt28t\") pod \"6629b726-b52a-4100-9bfd-ac8f4faca253\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") "
Nov 04 07:42:51.273334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273307    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tmp-dir\") pod \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") "
Nov 04 07:42:51.273662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273336    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-home" pod "6629b726-b52a-4100-9bfd-ac8f4faca253" (UID: "6629b726-b52a-4100-9bfd-ac8f4faca253"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:51.273662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273343    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-home\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-home\") pod \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\" (UID: \"b915520d-c11d-402e-b0ff-c01dfd5cdc4f\") "
Nov 04 07:42:51.273662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273389    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-model-cache" pod "6629b726-b52a-4100-9bfd-ac8f4faca253" (UID: "6629b726-b52a-4100-9bfd-ac8f4faca253"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:51.273662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273424    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6629b726-b52a-4100-9bfd-ac8f4faca253-tls-certs\" (UniqueName: \"kubernetes.io/secret/6629b726-b52a-4100-9bfd-ac8f4faca253-tls-certs\") pod \"6629b726-b52a-4100-9bfd-ac8f4faca253\" (UID: \"6629b726-b52a-4100-9bfd-ac8f4faca253\") "
Nov 04 07:42:51.273662 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273635    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-model-cache" pod "b915520d-c11d-402e-b0ff-c01dfd5cdc4f" (UID: "b915520d-c11d-402e-b0ff-c01dfd5cdc4f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:51.273916 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273712    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tmp-dir" pod "b915520d-c11d-402e-b0ff-c01dfd5cdc4f" (UID: "b915520d-c11d-402e-b0ff-c01dfd5cdc4f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:51.273916 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273825    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-tmp-dir" pod "6629b726-b52a-4100-9bfd-ac8f4faca253" (UID: "6629b726-b52a-4100-9bfd-ac8f4faca253"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:51.273916 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273891    5626 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-home\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:51.273916 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273913    5626 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-model-cache\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:51.274123 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273929    5626 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-model-cache\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:51.274123 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273943    5626 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tmp-dir\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:51.274123 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.273991    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-home" pod "b915520d-c11d-402e-b0ff-c01dfd5cdc4f" (UID: "b915520d-c11d-402e-b0ff-c01dfd5cdc4f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:51.275837 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.275803    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tls-certs" pod "b915520d-c11d-402e-b0ff-c01dfd5cdc4f" (UID: "b915520d-c11d-402e-b0ff-c01dfd5cdc4f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:42:51.275971 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.275871    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-dshm" pod "b915520d-c11d-402e-b0ff-c01dfd5cdc4f" (UID: "b915520d-c11d-402e-b0ff-c01dfd5cdc4f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:51.276051 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.275969    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6629b726-b52a-4100-9bfd-ac8f4faca253-kube-api-access-pt28t" pod "6629b726-b52a-4100-9bfd-ac8f4faca253" (UID: "6629b726-b52a-4100-9bfd-ac8f4faca253"). InnerVolumeSpecName "kube-api-access-pt28t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:51.276051 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.276033    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6629b726-b52a-4100-9bfd-ac8f4faca253-tls-certs" pod "6629b726-b52a-4100-9bfd-ac8f4faca253" (UID: "6629b726-b52a-4100-9bfd-ac8f4faca253"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 07:42:51.276163 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.276038    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-kube-api-access-74s2f" pod "b915520d-c11d-402e-b0ff-c01dfd5cdc4f" (UID: "b915520d-c11d-402e-b0ff-c01dfd5cdc4f"). InnerVolumeSpecName "kube-api-access-74s2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:42:51.276998 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.276980    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-dshm" pod "6629b726-b52a-4100-9bfd-ac8f4faca253" (UID: "6629b726-b52a-4100-9bfd-ac8f4faca253"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:42:51.374642 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.374609    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74s2f\" (UniqueName: \"kubernetes.io/projected/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-kube-api-access-74s2f\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:51.374642 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.374637    5626 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-tls-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:51.374642 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.374647    5626 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-tmp-dir\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:51.374874 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.374655    5626 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-dshm\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:51.374874 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.374664    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt28t\" (UniqueName: \"kubernetes.io/projected/6629b726-b52a-4100-9bfd-ac8f4faca253-kube-api-access-pt28t\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:51.374874 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.374673    5626 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b915520d-c11d-402e-b0ff-c01dfd5cdc4f-home\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:51.374874 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.374681    5626 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6629b726-b52a-4100-9bfd-ac8f4faca253-tls-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:51.374874 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:51.374688    5626 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6629b726-b52a-4100-9bfd-ac8f4faca253-dshm\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:42:52.092220 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.092187    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="646c74b6-7721-410a-bccf-004404e9c41a" path="/var/lib/kubelet/pods/646c74b6-7721-410a-bccf-004404e9c41a/volumes"
Nov 04 07:42:52.092649 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.092634    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" path="/var/lib/kubelet/pods/a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6/volumes"
Nov 04 07:42:52.093031 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.093019    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" path="/var/lib/kubelet/pods/eb5f0d1c-bf2b-4f0d-8556-a85713f541aa/volumes"
Nov 04 07:42:52.104204 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.104179    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"
Nov 04 07:42:52.104334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.104290    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw" event={"ID":"b915520d-c11d-402e-b0ff-c01dfd5cdc4f","Type":"ContainerDied","Data":"7c726e4f192c3af3001110b58faa9b36954e7fae49a5fb45514d23534027c6b6"}
Nov 04 07:42:52.104334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.104328    5626 scope.go:122] "RemoveContainer" containerID="a795dc778676f539d477b36173eb1d46bae773433bce65c70ca87771d467acf3"
Nov 04 07:42:52.106141 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.106083    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6" event={"ID":"6629b726-b52a-4100-9bfd-ac8f4faca253","Type":"ContainerDied","Data":"f3f7e882db42e7ff847e1211e64d8fbb07d7c5747b265eaec5821774e186fb13"}
Nov 04 07:42:52.106141 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.106135    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"
Nov 04 07:42:52.118332 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.118306    5626 scope.go:122] "RemoveContainer" containerID="0702c6a1e941f4aa7bea140ebe7d8714eceb186904117e5c9b4fe6c1739f4a9b"
Nov 04 07:42:52.132983 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.132960    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"]
Nov 04 07:42:52.145944 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.145911    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-1f6ce4/premium-w4-1f6ce4-kserve-5db4676b8f-hfks6"]
Nov 04 07:42:52.159012 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.158992    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"]
Nov 04 07:42:52.164068 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:52.164043    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-1f6ce4/simulator-w4-1f6ce4-kserve-55b4489cdd-pc4dw"]
Nov 04 07:42:54.102035 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:54.102001    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" path="/var/lib/kubelet/pods/6629b726-b52a-4100-9bfd-ac8f4faca253/volumes"
Nov 04 07:42:54.102425 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:54.102410    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" path="/var/lib/kubelet/pods/b915520d-c11d-402e-b0ff-c01dfd5cdc4f/volumes"
Nov 04 07:42:55.498780 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:42:55.498737    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm" podUID="c5e98a2b-fc18-4ec7-9fc0-34330c4fc120" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:43:00.051372 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051335    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"]
Nov 04 07:43:00.051806 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051789    5626 cpu_manager.go:414] "containerMap: removing container" podUID="41718dd1-af1a-4e5b-953a-f9734b3f3483" containerName="payload-processing"
Nov 04 07:43:00.051869 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051807    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41718dd1-af1a-4e5b-953a-f9734b3f3483" containerName="payload-processing"
Nov 04 07:43:00.051869 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051823    5626 cpu_manager.go:414] "containerMap: removing container" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerName="istio-proxy"
Nov 04 07:43:00.051869 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051828    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerName="istio-proxy"
Nov 04 07:43:00.051869 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051841    5626 cpu_manager.go:414] "containerMap: removing container" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main"
Nov 04 07:43:00.051869 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051857    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main"
Nov 04 07:43:00.052019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051878    5626 cpu_manager.go:414] "containerMap: removing container" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main"
Nov 04 07:43:00.052019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051886    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main"
Nov 04 07:43:00.052019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051901    5626 cpu_manager.go:414] "containerMap: removing container" podUID="646c74b6-7721-410a-bccf-004404e9c41a" containerName="istio-proxy"
Nov 04 07:43:00.052019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051905    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="646c74b6-7721-410a-bccf-004404e9c41a" containerName="istio-proxy"
Nov 04 07:43:00.052019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051913    5626 cpu_manager.go:414] "containerMap: removing container" podUID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" containerName="istio-proxy"
Nov 04 07:43:00.052019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051918    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" containerName="istio-proxy"
Nov 04 07:43:00.052019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051927    5626 cpu_manager.go:414] "containerMap: removing container" podUID="f8f994c3-eace-4fe2-8094-ccfbf67f6684" containerName="revoke-keys"
Nov 04 07:43:00.052019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.051931    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8f994c3-eace-4fe2-8094-ccfbf67f6684" containerName="revoke-keys"
Nov 04 07:43:00.052019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.052000    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="41718dd1-af1a-4e5b-953a-f9734b3f3483" containerName="payload-processing"
Nov 04 07:43:00.052019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.052010    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb5f0d1c-bf2b-4f0d-8556-a85713f541aa" containerName="istio-proxy"
Nov 04 07:43:00.052019 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.052018    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="6629b726-b52a-4100-9bfd-ac8f4faca253" containerName="main"
Nov 04 07:43:00.052310 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.052028    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="b915520d-c11d-402e-b0ff-c01dfd5cdc4f" containerName="main"
Nov 04 07:43:00.052310 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.052039    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="646c74b6-7721-410a-bccf-004404e9c41a" containerName="istio-proxy"
Nov 04 07:43:00.052310 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.052044    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2b86fa0-09d3-49d8-b4dd-d6c356d41cf6" containerName="istio-proxy"
Nov 04 07:43:00.052310 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.052051    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8f994c3-eace-4fe2-8094-ccfbf67f6684" containerName="revoke-keys"
Nov 04 07:43:00.056791 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.056772    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.061239 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.061218    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-1f6ce4-openshift-default-dockercfg-vncb8\""
Nov 04 07:43:00.078891 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.078867    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"]
Nov 04 07:43:00.157971 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.157943    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istiod-ca-cert\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.158114 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.158007    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-certs\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.158114 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.158030    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-podinfo\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.158114 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.158110    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-token\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.158228 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.158161    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-data\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.158228 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.158182    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq67d\" (UniqueName: \"kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-kube-api-access-lq67d\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.158292 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.158263    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-socket\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.158329 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.158298    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-credential-socket\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.158409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.158387    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-envoy\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259218 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259178    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-data\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259218 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259227    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq67d\" (UniqueName: \"kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-kube-api-access-lq67d\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259489 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259248    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-socket\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259489 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259278    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-credential-socket\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259489 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259312    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-envoy\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259489 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259338    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istiod-ca-cert\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259489 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259417    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-certs\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259489 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259436    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-podinfo\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259489 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259464    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-token\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259822 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259676    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-data\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259822 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259720    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-socket\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259822 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259779    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-credential-socket\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.259922 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.259824    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-certs\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.260142 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.260113    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istiod-ca-cert\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.261848 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.261823    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-envoy\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.261947 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.261933    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-podinfo\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.271996 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.271975    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-token\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.272200 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.272183    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq67d\" (UniqueName: \"kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-kube-api-access-lq67d\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.369924 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.369845    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:00.658448 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.658421    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"]
Nov 04 07:43:00.659246 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:43:00.659223    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod365dbb4a_5cae_4719_9a3e_dcf721e8baec.slice/crio-7373aa1d85a9dff9d6500555bc193c1be947f446858f1f96c322dfb5b8713387 WatchSource:0}: Error finding container 7373aa1d85a9dff9d6500555bc193c1be947f446858f1f96c322dfb5b8713387: Status 404 returned error can't find the container with id 7373aa1d85a9dff9d6500555bc193c1be947f446858f1f96c322dfb5b8713387
Nov 04 07:43:00.661514 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.661478    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:43:00.661593 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.661553    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:43:00.661593 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:00.661586    5626 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 04 07:43:01.168863 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:01.168826    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb" event={"ID":"365dbb4a-5cae-4719-9a3e-dcf721e8baec","Type":"ContainerStarted","Data":"2781df22754caf685d0aa18ea9c2023d2a3d436ed076f19f2fa5354cb1f9adc4"}
Nov 04 07:43:01.169245 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:01.168892    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb" event={"ID":"365dbb4a-5cae-4719-9a3e-dcf721e8baec","Type":"ContainerStarted","Data":"7373aa1d85a9dff9d6500555bc193c1be947f446858f1f96c322dfb5b8713387"}
Nov 04 07:43:01.195405 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:01.195318    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb" podStartSLOduration=1.195300732 podStartE2EDuration="1.195300732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:43:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:43:01.195257202 +0000 UTC m=+1603.217472465" watchObservedRunningTime="2026-11-04 07:43:01.195300732 +0000 UTC m=+1603.217515996"
Nov 04 07:43:01.370968 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:01.370935    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:01.372590 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:01.372563    5626 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 04 07:43:01.372731 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:01.372610    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb" podUID="365dbb4a-5cae-4719-9a3e-dcf721e8baec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 04 07:43:02.370394 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:02.370344    5626 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 04 07:43:02.370774 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:02.370411    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb" podUID="365dbb4a-5cae-4719-9a3e-dcf721e8baec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 04 07:43:03.370289 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:03.370244    5626 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 04 07:43:03.370528 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:03.370309    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb" podUID="365dbb4a-5cae-4719-9a3e-dcf721e8baec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 04 07:43:04.370598 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:04.370564    5626 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 04 07:43:04.370987 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:04.370617    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb" podUID="365dbb4a-5cae-4719-9a3e-dcf721e8baec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 04 07:43:05.370350 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:05.370312    5626 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 04 07:43:05.370530 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:05.370426    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb" podUID="365dbb4a-5cae-4719-9a3e-dcf721e8baec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 04 07:43:05.498888 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:05.498854    5626 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm" podUID="c5e98a2b-fc18-4ec7-9fc0-34330c4fc120" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 07:43:06.185946 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:43:06.185911    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:43:06.370943 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:06.370909    5626 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 04 07:43:06.371115 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:06.370975    5626 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb" podUID="365dbb4a-5cae-4719-9a3e-dcf721e8baec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 04 07:43:07.374841 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:07.374810    5626 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:07.375403 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:07.375099    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:07.375800 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:07.375782    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:07.716968 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:07.716944    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"
Nov 04 07:43:07.832928 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:07.832899    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-plugins-config-volume\") pod \"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120\" (UID: \"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120\") "
Nov 04 07:43:07.833157 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:07.832952    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-kube-api-access-wxwct\" (UniqueName: \"kubernetes.io/projected/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-kube-api-access-wxwct\") pod \"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120\" (UID: \"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120\") "
Nov 04 07:43:07.833290 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:07.833265    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-plugins-config-volume" pod "c5e98a2b-fc18-4ec7-9fc0-34330c4fc120" (UID: "c5e98a2b-fc18-4ec7-9fc0-34330c4fc120"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:43:07.834976 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:07.834956    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-kube-api-access-wxwct" pod "c5e98a2b-fc18-4ec7-9fc0-34330c4fc120" (UID: "c5e98a2b-fc18-4ec7-9fc0-34330c4fc120"). InnerVolumeSpecName "kube-api-access-wxwct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:43:07.934425 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:07.934333    5626 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-plugins-config-volume\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:43:07.934425 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:07.934387    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxwct\" (UniqueName: \"kubernetes.io/projected/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120-kube-api-access-wxwct\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:43:08.215032 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:08.214946    5626 generic.go:356] "Generic (PLEG): container finished" podID="c5e98a2b-fc18-4ec7-9fc0-34330c4fc120" containerID="4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829" exitCode=0
Nov 04 07:43:08.215032 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:08.214994    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"
Nov 04 07:43:08.215032 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:08.215019    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm" event={"ID":"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120","Type":"ContainerDied","Data":"4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829"}
Nov 04 07:43:08.215281 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:08.215079    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm" event={"ID":"c5e98a2b-fc18-4ec7-9fc0-34330c4fc120","Type":"ContainerDied","Data":"ffc24a7fec303654e8eae315d84793356521e47d50c197c04d39958314f23bc3"}
Nov 04 07:43:08.215281 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:08.215111    5626 scope.go:122] "RemoveContainer" containerID="4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829"
Nov 04 07:43:08.224590 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:08.224571    5626 scope.go:122] "RemoveContainer" containerID="4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829"
Nov 04 07:43:08.224862 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:43:08.224840    5626 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829\": container with ID starting with 4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829 not found: ID does not exist" containerID="4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829"
Nov 04 07:43:08.224905 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:08.224872    5626 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829"} err="failed to get container status \"4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829\": rpc error: code = NotFound desc = could not find container \"4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829\": container with ID starting with 4e90afeb552d52890bb78ee309f26434342caa2a84158f9f616fb8ae94c33829 not found: ID does not exist"
Nov 04 07:43:08.241753 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:08.241654    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"]
Nov 04 07:43:08.243113 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:08.243086    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm"]
Nov 04 07:43:10.092431 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:10.092394    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5e98a2b-fc18-4ec7-9fc0-34330c4fc120" path="/var/lib/kubelet/pods/c5e98a2b-fc18-4ec7-9fc0-34330c4fc120/volumes"
Nov 04 07:43:15.143254 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:15.143213    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"]
Nov 04 07:43:15.143917 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:15.143460    5626 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb" podUID="365dbb4a-5cae-4719-9a3e-dcf721e8baec" containerName="istio-proxy" containerID="cri-o://2781df22754caf685d0aa18ea9c2023d2a3d436ed076f19f2fa5354cb1f9adc4" gracePeriod=30
Nov 04 07:43:16.156621 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:16.156587    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7cc7fd6fd6-lmgkl_84888547-b6a2-4dc5-9a93-ac851db37e0e/authorino/0.log"
Nov 04 07:43:18.535156 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:18.535121    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84bbcb8bf9-hjnv7_507645ed-1327-42b2-98fe-f059d5e3eeea/manager/0.log"
Nov 04 07:43:19.593605 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:19.593572    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-msppb_2d27b8cd-6f7a-4657-9105-ccc5d7b05652/server/0.log"
Nov 04 07:43:20.297467 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.297433    5626 generic.go:356] "Generic (PLEG): container finished" podID="365dbb4a-5cae-4719-9a3e-dcf721e8baec" containerID="2781df22754caf685d0aa18ea9c2023d2a3d436ed076f19f2fa5354cb1f9adc4" exitCode=0
Nov 04 07:43:20.297625 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.297505    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb" event={"ID":"365dbb4a-5cae-4719-9a3e-dcf721e8baec","Type":"ContainerDied","Data":"2781df22754caf685d0aa18ea9c2023d2a3d436ed076f19f2fa5354cb1f9adc4"}
Nov 04 07:43:20.398640 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.398612    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:20.463225 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463140    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-data\") pod \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") "
Nov 04 07:43:20.463225 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463175    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-socket\") pod \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") "
Nov 04 07:43:20.463225 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463225    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-podinfo\") pod \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") "
Nov 04 07:43:20.463552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463250    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-kube-api-access-lq67d\" (UniqueName: \"kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-kube-api-access-lq67d\") pod \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") "
Nov 04 07:43:20.463552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463280    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-credential-socket\") pod \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") "
Nov 04 07:43:20.463552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463322    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-token\" (UniqueName: \"kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-token\") pod \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") "
Nov 04 07:43:20.463552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463389    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istiod-ca-cert\") pod \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") "
Nov 04 07:43:20.463552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463417    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-envoy\") pod \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") "
Nov 04 07:43:20.463552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463450    5626 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-certs\") pod \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\" (UID: \"365dbb4a-5cae-4719-9a3e-dcf721e8baec\") "
Nov 04 07:43:20.463552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463486    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-data" pod "365dbb4a-5cae-4719-9a3e-dcf721e8baec" (UID: "365dbb4a-5cae-4719-9a3e-dcf721e8baec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:43:20.463894 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463573    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-socket" pod "365dbb4a-5cae-4719-9a3e-dcf721e8baec" (UID: "365dbb4a-5cae-4719-9a3e-dcf721e8baec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:43:20.463894 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463615    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-credential-socket" pod "365dbb4a-5cae-4719-9a3e-dcf721e8baec" (UID: "365dbb4a-5cae-4719-9a3e-dcf721e8baec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:43:20.463894 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463773    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-certs" pod "365dbb4a-5cae-4719-9a3e-dcf721e8baec" (UID: "365dbb4a-5cae-4719-9a3e-dcf721e8baec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:43:20.463894 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463850    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istiod-ca-cert" pod "365dbb4a-5cae-4719-9a3e-dcf721e8baec" (UID: "365dbb4a-5cae-4719-9a3e-dcf721e8baec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 07:43:20.463894 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463876    5626 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-credential-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:43:20.463894 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463893    5626 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-certs\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:43:20.464118 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463905    5626 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-data\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:43:20.464118 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.463916    5626 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-workload-socket\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:43:20.465630 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.465599    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-token" pod "365dbb4a-5cae-4719-9a3e-dcf721e8baec" (UID: "365dbb4a-5cae-4719-9a3e-dcf721e8baec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:43:20.465766 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.465650    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-envoy" pod "365dbb4a-5cae-4719-9a3e-dcf721e8baec" (UID: "365dbb4a-5cae-4719-9a3e-dcf721e8baec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 07:43:20.465954 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.465938    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-kube-api-access-lq67d" pod "365dbb4a-5cae-4719-9a3e-dcf721e8baec" (UID: "365dbb4a-5cae-4719-9a3e-dcf721e8baec"). InnerVolumeSpecName "kube-api-access-lq67d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 07:43:20.465997 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.465939    5626 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-podinfo" pod "365dbb4a-5cae-4719-9a3e-dcf721e8baec" (UID: "365dbb4a-5cae-4719-9a3e-dcf721e8baec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 07:43:20.564587 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.564548    5626 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-podinfo\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:43:20.564587 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.564580    5626 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq67d\" (UniqueName: \"kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-kube-api-access-lq67d\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:43:20.564587 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.564593    5626 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-token\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:43:20.564825 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.564603    5626 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istiod-ca-cert\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:43:20.564825 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:20.564612    5626 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/365dbb4a-5cae-4719-9a3e-dcf721e8baec-istio-envoy\") on node \"ip-10-0-142-197.us-east-2.compute.internal\" DevicePath \"\""
Nov 04 07:43:21.305428 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:21.305394    5626 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"
Nov 04 07:43:21.305428 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:21.305358    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb" event={"ID":"365dbb4a-5cae-4719-9a3e-dcf721e8baec","Type":"ContainerDied","Data":"7373aa1d85a9dff9d6500555bc193c1be947f446858f1f96c322dfb5b8713387"}
Nov 04 07:43:21.306002 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:21.305464    5626 scope.go:122] "RemoveContainer" containerID="2781df22754caf685d0aa18ea9c2023d2a3d436ed076f19f2fa5354cb1f9adc4"
Nov 04 07:43:21.330949 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:21.330912    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"]
Nov 04 07:43:21.341001 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:21.337250    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb"]
Nov 04 07:43:22.092377 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:22.092335    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="365dbb4a-5cae-4719-9a3e-dcf721e8baec" path="/var/lib/kubelet/pods/365dbb4a-5cae-4719-9a3e-dcf721e8baec/volumes"
Nov 04 07:43:24.128740 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:24.128708    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7cc7fd6fd6-lmgkl_84888547-b6a2-4dc5-9a93-ac851db37e0e/authorino/0.log"
Nov 04 07:43:35.530511 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:35.530428    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84bbcb8bf9-hjnv7_507645ed-1327-42b2-98fe-f059d5e3eeea/manager/0.log"
Nov 04 07:43:36.172814 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:43:36.172755    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:43:36.621578 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:36.621551    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-msppb_2d27b8cd-6f7a-4657-9105-ccc5d7b05652/server/0.log"
Nov 04 07:43:38.121858 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:38.121829    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-03bf3578-g4zm2_a8b5311c-e2ce-4717-9009-8e53b9488740/revoke-keys/0.log"
Nov 04 07:43:38.294028 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:38.294000    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0e9be1a1-5ndst_d4be153f-9b45-4b53-acea-1b2bfac28202/revoke-keys/0.log"
Nov 04 07:43:38.478083 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:38.478010    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-19f7830f-vwxqb_a5e69a0d-c6ca-48e1-988f-76fca35a1183/revoke-keys/0.log"
Nov 04 07:43:38.656499 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:38.656471    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-1c7e5d0b-4c8qh_e30c2240-1bc9-4538-8f44-8cf3c22c1552/revoke-keys/0.log"
Nov 04 07:43:38.844693 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:38.844664    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-1e59041f-cc5hn_4e15c935-b53a-4307-ac3b-66a07729f311/revoke-keys/0.log"
Nov 04 07:43:39.021639 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:39.021613    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-256bd16c-prww5_5ca75d4f-c0f9-4e86-bb1d-d455f909acb8/revoke-keys/0.log"
Nov 04 07:43:39.053409 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:39.053380    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"]
Nov 04 07:43:39.059193 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:39.059165    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3eb9c5e-zwmv8"]
Nov 04 07:43:39.203045 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:39.202967    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-2883a0a1-xqt6h_4d1b8f8c-2e42-4274-b95d-e5c682e92079/revoke-keys/0.log"
Nov 04 07:43:39.379582 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:39.379555    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-2a2a6a3e-8wh97_31ee0f07-60d1-4aaf-a327-feafad46276f/revoke-keys/0.log"
Nov 04 07:43:39.545589 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:39.545563    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-39d3b012-4f66t_f5aed379-b54c-4513-a0e6-20973954e19a/revoke-keys/0.log"
Nov 04 07:43:39.721830 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:39.721799    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-925c0037-pvz85_4b66971a-d867-4a9f-87c7-78032e90047a/revoke-keys/0.log"
Nov 04 07:43:39.898581 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:39.898512    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-9ca1547e-msm67_f8f994c3-eace-4fe2-8094-ccfbf67f6684/revoke-keys/0.log"
Nov 04 07:43:40.092549 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:40.092517    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c968ff1-4226-4571-8726-0c7f2f463e11" path="/var/lib/kubelet/pods/8c968ff1-4226-4571-8726-0c7f2f463e11/volumes"
Nov 04 07:43:40.094673 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:40.094650    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-a4bede4a-5gfvp_700217a7-2875-4c01-a556-f0741045b436/revoke-keys/0.log"
Nov 04 07:43:40.285144 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:40.285113    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-b3518b89-p2s5j_a81809e4-7c2d-4c52-98a0-4e77f0124045/revoke-keys/0.log"
Nov 04 07:43:40.456334 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:40.456304    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-c039095a-thj7r_b9798d72-17cb-421b-836d-942de901c436/revoke-keys/0.log"
Nov 04 07:43:40.794956 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:40.794932    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-c3fcc35a-jjmf2_216d4709-2b24-4351-bea6-b2d686dab302/revoke-keys/0.log"
Nov 04 07:43:40.963216 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:40.963184    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-d8e11c6e-2hvw5_0e3d2c37-c22a-41d5-985b-6ab478e9528a/revoke-keys/0.log"
Nov 04 07:43:41.134332 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:41.134261    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-e0ee3629-fszdp_75bd9dcf-c567-4dad-a161-4d045f108bc2/revoke-keys/0.log"
Nov 04 07:43:43.156463 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:43.156433    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7cc7fd6fd6-lmgkl_84888547-b6a2-4dc5-9a93-ac851db37e0e/authorino/0.log"
Nov 04 07:43:46.415847 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:46.415803    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589fcc7ff4-h6ggf_78a42108-2b93-415c-a7e2-075603f71c38/kube-auth-proxy/0.log"
Nov 04 07:43:46.791966 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:46.791930    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-788ls_5be66e14-e024-4afe-8e07-9cd3cc138ea5/istio-proxy/0.log"
Nov 04 07:43:47.324299 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:47.324266    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-g5rgr_614988be-64cf-4a39-b201-ff69a7f7fb19/istio-proxy/0.log"
Nov 04 07:43:48.052067 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:48.052034    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-rxf9k_71b5a865-3278-44ff-a614-7c6c4d0c42c5/istio-proxy/0.log"
Nov 04 07:43:48.233123 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:48.233095    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-xgjgh_7ba40135-db97-4238-8bb8-f1fd581a1615/istio-proxy/1.log"
Nov 04 07:43:49.614427 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:49.614398    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-xgjgh_7ba40135-db97-4238-8bb8-f1fd581a1615/istio-proxy/0.log"
Nov 04 07:43:50.043739 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:50.043707    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"]
Nov 04 07:43:50.052504 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:50.052476    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-39d3b012-4f66t"]
Nov 04 07:43:50.092180 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:50.092152    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5aed379-b54c-4513-a0e6-20973954e19a" path="/var/lib/kubelet/pods/f5aed379-b54c-4513-a0e6-20973954e19a/volumes"
Nov 04 07:43:50.313347 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:50.313271    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-v2t9p_f2fa15d8-82b1-4d8f-a541-574254719b50/main/0.log"
Nov 04 07:43:50.662120 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:50.662040    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-226jt_d698e45f-e68d-4c6c-99d2-ab4a646437d0/main/0.log"
Nov 04 07:43:50.848325 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:50.848279    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j_5d1867e7-cf85-4418-9a27-501735d097bd/main/0.log"
Nov 04 07:43:51.395833 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:43:51.395797    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn_573aaebc-ca66-4bf2-84a5-2057a5e77b87/main/0.log"
Nov 04 07:44:01.736586 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.736549    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zs2zc/must-gather-ksvzs"]
Nov 04 07:44:01.737000 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.736953    5626 cpu_manager.go:414] "containerMap: removing container" podUID="c5e98a2b-fc18-4ec7-9fc0-34330c4fc120" containerName="payload-pre-processing"
Nov 04 07:44:01.737000 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.736964    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5e98a2b-fc18-4ec7-9fc0-34330c4fc120" containerName="payload-pre-processing"
Nov 04 07:44:01.737000 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.736981    5626 cpu_manager.go:414] "containerMap: removing container" podUID="365dbb4a-5cae-4719-9a3e-dcf721e8baec" containerName="istio-proxy"
Nov 04 07:44:01.737000 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.736986    5626 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="365dbb4a-5cae-4719-9a3e-dcf721e8baec" containerName="istio-proxy"
Nov 04 07:44:01.737136 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.737044    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="365dbb4a-5cae-4719-9a3e-dcf721e8baec" containerName="istio-proxy"
Nov 04 07:44:01.737136 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.737059    5626 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5e98a2b-fc18-4ec7-9fc0-34330c4fc120" containerName="payload-pre-processing"
Nov 04 07:44:01.741673 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.741655    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zs2zc/must-gather-ksvzs"
Nov 04 07:44:01.751331 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.751309    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zs2zc\"/\"kube-root-ca.crt\""
Nov 04 07:44:01.751449 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.751333    5626 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zs2zc\"/\"openshift-service-ca.crt\""
Nov 04 07:44:01.753113 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.753094    5626 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zs2zc\"/\"default-dockercfg-6mkpg\""
Nov 04 07:44:01.755999 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.755975    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zs2zc/must-gather-ksvzs"]
Nov 04 07:44:01.757759 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.757737    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d8cbce8b-ddce-41e9-96d6-6102243bd495-must-gather-output\") pod \"must-gather-ksvzs\" (UID: \"d8cbce8b-ddce-41e9-96d6-6102243bd495\") " pod="openshift-must-gather-zs2zc/must-gather-ksvzs"
Nov 04 07:44:01.757857 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.757804    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnk2f\" (UniqueName: \"kubernetes.io/projected/d8cbce8b-ddce-41e9-96d6-6102243bd495-kube-api-access-bnk2f\") pod \"must-gather-ksvzs\" (UID: \"d8cbce8b-ddce-41e9-96d6-6102243bd495\") " pod="openshift-must-gather-zs2zc/must-gather-ksvzs"
Nov 04 07:44:01.858552 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.858519    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnk2f\" (UniqueName: \"kubernetes.io/projected/d8cbce8b-ddce-41e9-96d6-6102243bd495-kube-api-access-bnk2f\") pod \"must-gather-ksvzs\" (UID: \"d8cbce8b-ddce-41e9-96d6-6102243bd495\") " pod="openshift-must-gather-zs2zc/must-gather-ksvzs"
Nov 04 07:44:01.858730 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.858591    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d8cbce8b-ddce-41e9-96d6-6102243bd495-must-gather-output\") pod \"must-gather-ksvzs\" (UID: \"d8cbce8b-ddce-41e9-96d6-6102243bd495\") " pod="openshift-must-gather-zs2zc/must-gather-ksvzs"
Nov 04 07:44:01.858935 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.858915    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d8cbce8b-ddce-41e9-96d6-6102243bd495-must-gather-output\") pod \"must-gather-ksvzs\" (UID: \"d8cbce8b-ddce-41e9-96d6-6102243bd495\") " pod="openshift-must-gather-zs2zc/must-gather-ksvzs"
Nov 04 07:44:01.875550 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:01.875522    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnk2f\" (UniqueName: \"kubernetes.io/projected/d8cbce8b-ddce-41e9-96d6-6102243bd495-kube-api-access-bnk2f\") pod \"must-gather-ksvzs\" (UID: \"d8cbce8b-ddce-41e9-96d6-6102243bd495\") " pod="openshift-must-gather-zs2zc/must-gather-ksvzs"
Nov 04 07:44:02.048565 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:02.048536    5626 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"]
Nov 04 07:44:02.050747 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:02.050727    5626 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2a2a6a3e-8wh97"]
Nov 04 07:44:02.056449 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:02.056430    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zs2zc/must-gather-ksvzs"
Nov 04 07:44:02.092898 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:02.092868    5626 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31ee0f07-60d1-4aaf-a327-feafad46276f" path="/var/lib/kubelet/pods/31ee0f07-60d1-4aaf-a327-feafad46276f/volumes"
Nov 04 07:44:02.348932 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:02.348902    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zs2zc/must-gather-ksvzs"]
Nov 04 07:44:02.349913 ip-10-0-142-197 kubenswrapper[5626]: W1104 07:44:02.349888    5626 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8cbce8b_ddce_41e9_96d6_6102243bd495.slice/crio-5bd7a2b2ce2d1a440de7dddd11692592f8fd2c6e51f1a17e390f3de7af801ab5 WatchSource:0}: Error finding container 5bd7a2b2ce2d1a440de7dddd11692592f8fd2c6e51f1a17e390f3de7af801ab5: Status 404 returned error can't find the container with id 5bd7a2b2ce2d1a440de7dddd11692592f8fd2c6e51f1a17e390f3de7af801ab5
Nov 04 07:44:02.351906 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:02.351888    5626 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 07:44:02.540321 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:02.540287    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zs2zc/must-gather-ksvzs" event={"ID":"d8cbce8b-ddce-41e9-96d6-6102243bd495","Type":"ContainerStarted","Data":"5bd7a2b2ce2d1a440de7dddd11692592f8fd2c6e51f1a17e390f3de7af801ab5"}
Nov 04 07:44:03.554421 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:03.554297    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zs2zc/must-gather-ksvzs" event={"ID":"d8cbce8b-ddce-41e9-96d6-6102243bd495","Type":"ContainerStarted","Data":"ba2b914afd8e5af840311bfdc22d6e48c7d3894195f14c8969d2123f6cfc650a"}
Nov 04 07:44:03.554421 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:03.554389    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zs2zc/must-gather-ksvzs" event={"ID":"d8cbce8b-ddce-41e9-96d6-6102243bd495","Type":"ContainerStarted","Data":"ffb094caada265bfd7018e080b49221ec073a65dad75c742996a5606b727ff70"}
Nov 04 07:44:03.586123 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:03.586072    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-zs2zc/must-gather-ksvzs" podStartSLOduration=1.651412364 podStartE2EDuration="2.586058665s" totalImagesPullingTime="934.646301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:44:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:44:03.581157779 +0000 UTC m=+1665.603373039" watchObservedRunningTime="2026-11-04 07:44:03.586058665 +0000 UTC m=+1665.608273925"
Nov 04 07:44:05.490388 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:05.489616    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dxhns_bd7ea631-0ca9-4829-8408-8352e1d1bc7f/global-pull-secret-syncer/0.log"
Nov 04 07:44:05.966273 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:05.966231    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-knhlk_13da5ce6-d81c-49c2-b92b-ffb77d6a2e38/konnectivity-agent/0.log"
Nov 04 07:44:06.131766 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:06.131732    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-197.us-east-2.compute.internal_cec163fb92d8094aef96320b2516e2df/haproxy/0.log"
Nov 04 07:44:06.182395 ip-10-0-142-197 kubenswrapper[5626]: E1104 07:44:06.181261    5626 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5zbxq" containerName="extractor"
Nov 04 07:44:09.131473 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:09.131436    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84bbcb8bf9-hjnv7_507645ed-1327-42b2-98fe-f059d5e3eeea/manager/0.log"
Nov 04 07:44:09.593052 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:09.591404    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-msppb_2d27b8cd-6f7a-4657-9105-ccc5d7b05652/server/0.log"
Nov 04 07:44:11.113142 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:11.113107    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84bbcb8bf9-hjnv7_507645ed-1327-42b2-98fe-f059d5e3eeea/manager/0.log"
Nov 04 07:44:11.450669 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:11.449918    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-msppb_2d27b8cd-6f7a-4657-9105-ccc5d7b05652/server/0.log"
Nov 04 07:44:12.629393 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:12.629195    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7cc7fd6fd6-lmgkl_84888547-b6a2-4dc5-9a93-ac851db37e0e/authorino/0.log"
Nov 04 07:44:12.931604 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:12.931529    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84bbcb8bf9-hjnv7_507645ed-1327-42b2-98fe-f059d5e3eeea/manager/0.log"
Nov 04 07:44:13.410440 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:13.410274    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-msppb_2d27b8cd-6f7a-4657-9105-ccc5d7b05652/server/0.log"
Nov 04 07:44:15.089085 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.089053    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-6qqhd_13b111e6-49a0-45e4-bb33-a49c5dc4d439/kube-state-metrics/0.log"
Nov 04 07:44:15.124048 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.124017    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-6qqhd_13b111e6-49a0-45e4-bb33-a49c5dc4d439/kube-rbac-proxy-main/0.log"
Nov 04 07:44:15.169293 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.169265    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-6qqhd_13b111e6-49a0-45e4-bb33-a49c5dc4d439/kube-rbac-proxy-self/0.log"
Nov 04 07:44:15.220566 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.220534    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-76495f7c59-rsjg4_0ef74e4a-dee2-4554-be57-0eda5677ebf2/metrics-server/0.log"
Nov 04 07:44:15.292834 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.292774    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-2xqz7_2904aa1a-7fd1-43c5-922a-2a0a85e2ab05/monitoring-plugin/0.log"
Nov 04 07:44:15.348006 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.347922    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bwnvs_fff3c48c-f8c5-4946-a95e-b5faedb4dab8/node-exporter/0.log"
Nov 04 07:44:15.426808 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.426776    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bwnvs_fff3c48c-f8c5-4946-a95e-b5faedb4dab8/kube-rbac-proxy/0.log"
Nov 04 07:44:15.430940 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.430915    5626 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"]
Nov 04 07:44:15.436913 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.436885    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.453316 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.453293    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"]
Nov 04 07:44:15.487218 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.487176    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bwnvs_fff3c48c-f8c5-4946-a95e-b5faedb4dab8/init-textfile/0.log"
Nov 04 07:44:15.517864 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.517821    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a607fe3a-10a7-42a1-be00-a5dca06fb948-proc\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.518054 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.517900    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a607fe3a-10a7-42a1-be00-a5dca06fb948-podres\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.518054 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.517964    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6h26\" (UniqueName: \"kubernetes.io/projected/a607fe3a-10a7-42a1-be00-a5dca06fb948-kube-api-access-k6h26\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.518054 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.518050    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a607fe3a-10a7-42a1-be00-a5dca06fb948-lib-modules\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.518629 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.518190    5626 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a607fe3a-10a7-42a1-be00-a5dca06fb948-sys\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.619760 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.619660    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a607fe3a-10a7-42a1-be00-a5dca06fb948-lib-modules\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.619951 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.619778    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a607fe3a-10a7-42a1-be00-a5dca06fb948-sys\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.619951 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.619800    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a607fe3a-10a7-42a1-be00-a5dca06fb948-lib-modules\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.619951 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.619812    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a607fe3a-10a7-42a1-be00-a5dca06fb948-proc\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.619951 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.619870    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a607fe3a-10a7-42a1-be00-a5dca06fb948-proc\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.619951 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.619883    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a607fe3a-10a7-42a1-be00-a5dca06fb948-podres\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.619951 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.619923    5626 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6h26\" (UniqueName: \"kubernetes.io/projected/a607fe3a-10a7-42a1-be00-a5dca06fb948-kube-api-access-k6h26\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.619951 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.619942    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a607fe3a-10a7-42a1-be00-a5dca06fb948-sys\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.620381 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.620040    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a607fe3a-10a7-42a1-be00-a5dca06fb948-podres\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.644871 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.644830    5626 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6h26\" (UniqueName: \"kubernetes.io/projected/a607fe3a-10a7-42a1-be00-a5dca06fb948-kube-api-access-k6h26\") pod \"perf-node-gather-daemonset-hh9vk\" (UID: \"a607fe3a-10a7-42a1-be00-a5dca06fb948\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.755589 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.755514    5626 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:15.882048 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.881940    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-k6tsc_fe2e0b0d-3d26-4d4d-90a6-ef499452293b/kube-rbac-proxy-main/0.log"
Nov 04 07:44:15.917509 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.917478    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-k6tsc_fe2e0b0d-3d26-4d4d-90a6-ef499452293b/kube-rbac-proxy-self/0.log"
Nov 04 07:44:15.953052 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:15.953002    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-k6tsc_fe2e0b0d-3d26-4d4d-90a6-ef499452293b/openshift-state-metrics/0.log"
Nov 04 07:44:16.025468 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.024877    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a1cfe38-f37a-49e8-a29a-16c6bdb44759/prometheus/0.log"
Nov 04 07:44:16.064138 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.064107    5626 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"]
Nov 04 07:44:16.073153 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.073132    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a1cfe38-f37a-49e8-a29a-16c6bdb44759/config-reloader/0.log"
Nov 04 07:44:16.139098 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.139031    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a1cfe38-f37a-49e8-a29a-16c6bdb44759/thanos-sidecar/0.log"
Nov 04 07:44:16.176664 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.176625    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a1cfe38-f37a-49e8-a29a-16c6bdb44759/kube-rbac-proxy-web/0.log"
Nov 04 07:44:16.220151 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.220122    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a1cfe38-f37a-49e8-a29a-16c6bdb44759/kube-rbac-proxy/0.log"
Nov 04 07:44:16.257349 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.257318    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a1cfe38-f37a-49e8-a29a-16c6bdb44759/kube-rbac-proxy-thanos/0.log"
Nov 04 07:44:16.291925 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.291888    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a1cfe38-f37a-49e8-a29a-16c6bdb44759/init-config-reloader/0.log"
Nov 04 07:44:16.521051 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.520987    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589fcc7ff4-h6ggf_78a42108-2b93-415c-a7e2-075603f71c38/kube-auth-proxy/0.log"
Nov 04 07:44:16.599839 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.597412    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-788ls_5be66e14-e024-4afe-8e07-9cd3cc138ea5/istio-proxy/0.log"
Nov 04 07:44:16.692106 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.692078    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-g5rgr_614988be-64cf-4a39-b201-ff69a7f7fb19/istio-proxy/0.log"
Nov 04 07:44:16.705220 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.705185    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk" event={"ID":"a607fe3a-10a7-42a1-be00-a5dca06fb948","Type":"ContainerStarted","Data":"952761a13f115846e9aee82b817070e8aceacbd8a5f518fde485cde1777dc2eb"}
Nov 04 07:44:16.705393 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.705286    5626 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk" event={"ID":"a607fe3a-10a7-42a1-be00-a5dca06fb948","Type":"ContainerStarted","Data":"57dfd10056689134e86cea8dd7001b3ec14b7803a8c97eeda4e48522a1fce0a2"}
Nov 04 07:44:16.705393 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.705318    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:16.732939 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.732915    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64cb9dd857-hf26d_77e0a9a6-4d92-4f21-8ecf-7b791b33a730/thanos-query/0.log"
Nov 04 07:44:16.790352 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.790326    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64cb9dd857-hf26d_77e0a9a6-4d92-4f21-8ecf-7b791b33a730/kube-rbac-proxy-web/0.log"
Nov 04 07:44:16.836383 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.834012    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-rxf9k_71b5a865-3278-44ff-a614-7c6c4d0c42c5/istio-proxy/0.log"
Nov 04 07:44:16.843725 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.843700    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64cb9dd857-hf26d_77e0a9a6-4d92-4f21-8ecf-7b791b33a730/kube-rbac-proxy/0.log"
Nov 04 07:44:16.859802 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.859779    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-xgjgh_7ba40135-db97-4238-8bb8-f1fd581a1615/istio-proxy/0.log"
Nov 04 07:44:16.861458 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.861436    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-xgjgh_7ba40135-db97-4238-8bb8-f1fd581a1615/istio-proxy/1.log"
Nov 04 07:44:16.880397 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.877325    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64cb9dd857-hf26d_77e0a9a6-4d92-4f21-8ecf-7b791b33a730/prom-label-proxy/0.log"
Nov 04 07:44:16.906557 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.906530    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64cb9dd857-hf26d_77e0a9a6-4d92-4f21-8ecf-7b791b33a730/kube-rbac-proxy-rules/0.log"
Nov 04 07:44:16.947495 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:16.947468    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64cb9dd857-hf26d_77e0a9a6-4d92-4f21-8ecf-7b791b33a730/kube-rbac-proxy-metrics/0.log"
Nov 04 07:44:18.684926 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:18.684885    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7cc7fd6fd6-lmgkl_84888547-b6a2-4dc5-9a93-ac851db37e0e/authorino/0.log"
Nov 04 07:44:18.685471 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:18.685423    5626 scope.go:122] "RemoveContainer" containerID="aabdfcd4c12ce7ce6d3343867c9583795c4f69d1ea652a5cbd376bc9fcf4f84b"
Nov 04 07:44:21.929069 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:21.929022    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2w47v_52e5228a-5720-4bda-9af5-2c1a2d014923/dns/0.log"
Nov 04 07:44:21.970184 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:21.970155    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2w47v_52e5228a-5720-4bda-9af5-2c1a2d014923/kube-rbac-proxy/0.log"
Nov 04 07:44:22.154594 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:22.154567    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2zd55_f95c6e6a-6787-423e-8129-a62feb0a5156/dns-node-resolver/0.log"
Nov 04 07:44:22.724069 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:22.724040    5626 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk"
Nov 04 07:44:22.754144 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:22.754096    5626 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-hh9vk" podStartSLOduration=7.75408112 podStartE2EDuration="7.75408112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:44:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:44:16.730247186 +0000 UTC m=+1678.752462447" watchObservedRunningTime="2026-11-04 07:44:22.75408112 +0000 UTC m=+1684.776296380"
Nov 04 07:44:23.030438 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:23.030409    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2pxh9_3f1d3530-4c03-4633-92b1-9fc2ab954b9f/node-ca/0.log"
Nov 04 07:44:24.710385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:24.709776    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-597b95cb84-qpmxd_13ec8661-cef2-4782-a530-d781db896e56/cert-manager-controller/0.log"
Nov 04 07:44:24.752934 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:24.752904    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-cainjector-5c57bbd4fc-sqp55_d3939b90-6cf5-4733-a086-71a0393f707f/cert-manager-cainjector/0.log"
Nov 04 07:44:24.768405 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:24.768374    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-webhook-5dbbbf75cc-q9whl_07041b7d-8ce2-4dfc-933e-e4bb7866b9c4/cert-manager-webhook/0.log"
Nov 04 07:44:25.506668 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:25.506544    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-8f94f88d4-5nk76_b5d15c34-320f-4e4f-b159-fda262255cb7/manager/0.log"
Nov 04 07:44:25.528864 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:25.528833    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-8f94f88d4-5nk76_b5d15c34-320f-4e4f-b159-fda262255cb7/copy-manifests/0.log"
Nov 04 07:44:25.964963 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:25.964928    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589fcc7ff4-h6ggf_78a42108-2b93-415c-a7e2-075603f71c38/kube-auth-proxy/0.log"
Nov 04 07:44:26.099307 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:26.099273    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-788ls_5be66e14-e024-4afe-8e07-9cd3cc138ea5/istio-proxy/0.log"
Nov 04 07:44:26.220432 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:26.220302    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-g5rgr_614988be-64cf-4a39-b201-ff69a7f7fb19/istio-proxy/0.log"
Nov 04 07:44:26.320004 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:26.319968    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-84bbcb8bf9-hjnv7_507645ed-1327-42b2-98fe-f059d5e3eeea/manager/0.log"
Nov 04 07:44:26.384327 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:26.384296    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-rxf9k_71b5a865-3278-44ff-a614-7c6c4d0c42c5/istio-proxy/0.log"
Nov 04 07:44:26.420314 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:26.420272    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-xgjgh_7ba40135-db97-4238-8bb8-f1fd581a1615/istio-proxy/0.log"
Nov 04 07:44:26.422035 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:26.422010    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-xgjgh_7ba40135-db97-4238-8bb8-f1fd581a1615/istio-proxy/1.log"
Nov 04 07:44:26.753502 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:26.753469    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-msppb_2d27b8cd-6f7a-4657-9105-ccc5d7b05652/server/0.log"
Nov 04 07:44:27.807711 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:27.807685    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589fcc7ff4-h6ggf_78a42108-2b93-415c-a7e2-075603f71c38/kube-auth-proxy/0.log"
Nov 04 07:44:27.860152 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:27.860123    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-788ls_5be66e14-e024-4afe-8e07-9cd3cc138ea5/istio-proxy/0.log"
Nov 04 07:44:27.943817 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:27.943787    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-g5rgr_614988be-64cf-4a39-b201-ff69a7f7fb19/istio-proxy/0.log"
Nov 04 07:44:28.026841 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:28.026803    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-rxf9k_71b5a865-3278-44ff-a614-7c6c4d0c42c5/istio-proxy/0.log"
Nov 04 07:44:28.051070 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:28.051041    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-xgjgh_7ba40135-db97-4238-8bb8-f1fd581a1615/istio-proxy/0.log"
Nov 04 07:44:28.052956 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:28.052912    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-xgjgh_7ba40135-db97-4238-8bb8-f1fd581a1615/istio-proxy/1.log"
Nov 04 07:44:28.163124 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:28.163053    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f5wrt_24419200-a11c-445d-8a5b-d35e6473614f/serve-healthcheck-canary/0.log"
Nov 04 07:44:28.970950 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:28.970919    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5zbxq_723c3b52-1b88-45aa-9ab1-c0a712bf733e/kube-rbac-proxy/0.log"
Nov 04 07:44:28.999790 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:28.999721    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5zbxq_723c3b52-1b88-45aa-9ab1-c0a712bf733e/exporter/0.log"
Nov 04 07:44:29.035885 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:29.035850    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5zbxq_723c3b52-1b88-45aa-9ab1-c0a712bf733e/extractor/0.log"
Nov 04 07:44:29.440477 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:29.440452    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589fcc7ff4-h6ggf_78a42108-2b93-415c-a7e2-075603f71c38/kube-auth-proxy/0.log"
Nov 04 07:44:29.486191 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:29.486164    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-788ls_5be66e14-e024-4afe-8e07-9cd3cc138ea5/istio-proxy/0.log"
Nov 04 07:44:29.558192 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:29.558158    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-g5rgr_614988be-64cf-4a39-b201-ff69a7f7fb19/istio-proxy/0.log"
Nov 04 07:44:29.658142 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:29.658115    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-rxf9k_71b5a865-3278-44ff-a614-7c6c4d0c42c5/istio-proxy/0.log"
Nov 04 07:44:29.676215 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:29.676184    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-xgjgh_7ba40135-db97-4238-8bb8-f1fd581a1615/istio-proxy/0.log"
Nov 04 07:44:29.677972 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:29.677944    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-xgjgh_7ba40135-db97-4238-8bb8-f1fd581a1615/istio-proxy/1.log"
Nov 04 07:44:31.114585 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:31.114491    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-controller-manager-6d445bbc67-m4gfl_4fb3aedb-6fac-44fa-8823-d926fa0d71f9/manager/0.log"
Nov 04 07:44:31.169572 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:31.169538    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-operator-674cd59dc5-lrplm_5c70f97d-8ee2-4adb-9d82-6cf949ab02fd/jobset-operator/0.log"
Nov 04 07:44:31.460995 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:31.460929    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589fcc7ff4-h6ggf_78a42108-2b93-415c-a7e2-075603f71c38/kube-auth-proxy/0.log"
Nov 04 07:44:31.511919 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:31.511872    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-788ls_5be66e14-e024-4afe-8e07-9cd3cc138ea5/istio-proxy/0.log"
Nov 04 07:44:31.595051 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:31.595021    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-g5rgr_614988be-64cf-4a39-b201-ff69a7f7fb19/istio-proxy/0.log"
Nov 04 07:44:31.684020 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:31.682799    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-rxf9k_71b5a865-3278-44ff-a614-7c6c4d0c42c5/istio-proxy/0.log"
Nov 04 07:44:31.712646 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:31.712525    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-xgjgh_7ba40135-db97-4238-8bb8-f1fd581a1615/istio-proxy/0.log"
Nov 04 07:44:31.713714 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:31.713693    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-xgjgh_7ba40135-db97-4238-8bb8-f1fd581a1615/istio-proxy/1.log"
Nov 04 07:44:32.341251 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:32.341181    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-8f94f88d4-5nk76_b5d15c34-320f-4e4f-b159-fda262255cb7/manager/0.log"
Nov 04 07:44:32.380385 ip-10-0-142-197 kubenswrapper[5626]: I1104 07:44:32.377853    5626 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-8f94f88d4-5nk76_b5d15c34-320f-4e4f-b159-fda262255cb7/copy-manifests/0.log"